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(54) Displaying information 



(57) An enlarged image is displayed using a plurality 
of television sets. A television set serving as a master 
device and television sets serving as slave devices con- 
vert an input image into partial enlarged images and dis- 
play the resultant partial enlarged images so that the 



partial enlarged images displayed on the respective tel- 
evision sets form, as a whole, a complete enlarged full 
image. The master device and slave devices perform 
mutual authentication with each other. If the authentica- 
tion is successfully passed, the operation mode is set 
so that displaying of an enlarged image is allowed. 



FIG. 1A 



< 

CD 
CO 

CD 
Ol 

CO 




Q. 
LU 



Primed by Joovo, 75001 PARIS (FR) 



EP 1 326 436 A2 

Description 

BACKGROUND OF THE INVENTION 
5 1 . Field of the Invention 

[0001] The present invention relates to a display apparatus, a method of controlling a display device, a program, a 
storage medium, and a display system, and more particularly, to a display apparatus, a method of controlling a display 
device, a program, a storage medium, and a display system, which allow a plurality of display apparatuses so as to 
10 achieve a higher capability than can be achieved by a single display apparatus. 

2. Description of the Related Art 

[0002] In television sets, an image and an associated sound/voice are output in accordance with a received television 
15 broadcast signal. 

[0003] The conventional television sets are designed on the assumption that each television set is used separately 
from other television sets. If a user purchases a new television set, an old television set possessed by the user becomes 
unnecessary and, in many cases, the old television set is thrown out. 

[0004] When a plurality of television sets are used together, if a higher capability is achieved than can be achieved 
20 by a single television set, old television sets can be used usefully without being thrown out. 

[0005] In view of the above, an object of the present invention is to provide a technique of combining a plurality of 
television sets or display apparatuses so as to achieve a higher capability than can be achieved by a single television 
set or a single display apparatus. 

25 SUMMARY OF THE INVENTION 

[0006] The present invention provides a first display apparatus, connectable with one or more other display appa- 
ratuses and having display means for displaying an image in accordance with an input video signal, comprising: clas- 
sifying means for determining a class corresponding to a pixel of interest such that plural class reference pixels in the 

30 vicinity of the pixel of interest to be predicted are extracted from the input video signal and the class corresponding to 
the pixel of interest is determined from the extracted class reference pixels; prediction coefficient generation means 
for generating a prediction coefficient corresponding to the class determined by the classifying means; pixel prediction 
means for predicting the pixel of interest such that plural prediction reference pixels in the vicinity of the pixel of interest 
are extracted from the input video signal, and the pixel of interest is predicted by means of a prediction operation using 

35 the extracted plural prediction reference pixels and the prediction coefficient; and display control means for displaying 
an image including at least the pixel of interest on display means such that images displayed on the present display 
apparatus and said one or more other display apparatuses form, as a whole, a complete enlarged image of the image 
corresponding to the input video signal. 

[0007] The present invention provides a first display method for a display apparatus, connectable with one or more 

40 other display apparatuses, to display an image in accordance with an input video signal, comprising: a classification 
step, of determining a class corresponding to a pixel of interest such that plural class reference pixels in the vicinity of 
the pixel of interest to be predicted are extracted from the input video signal and the class corresponding to the pixel 
of interest is determined from the extracted class reference pixels; a prediction coefficient generation step for generating 
a prediction coefficient corresponding to the class determined in the classification step; a pixel prediction step for 

45 predicting the pixel of interest such that plural prediction reference pixels in the vicinity of the pixel of interest are 
extracted from the input video signal, and the pixel of interest is predicted by means of a prediction operation using 
the extracted plural prediction reference pixels and the prediction coefficient; and a display control step, of displaying 
an image including at least the pixel of interest on display means such that images displayed on the present display 
apparatus and said one or more other display apparatuses form, as a whole, a complete enlarged image of the image 

50 corresponding to the input video signal. 

[0008] The present invention provides a first program for causing a computer to control a display apparatus, which 
is connectable with one or more other display apparatuses, so as to display an image in accordance with an input 
video signal, said program comprising: a classification step, of determining a class corresponding to a pixel of interest 
such that plural class reference pixels in the vicinity of the pixel of interest to be predicted are extracted from the input 

55 video signal and the class corresponding to the pixel of interest is determined from the extracted class reference pixels; 
a prediction coefficient generation step for generating a prediction coefficient corresponding to the class determined 
in the classification step; a pixel prediction step for predicting the pixel of interest such that plural prediction reference 
pixels in the vicinity of the pixel of interest are extracted from the input video signal, and the pixel of interest is predicted 
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by means of a prediction operation using the extracted plural prediction reference pixels and the prediction coefficient; 
and a display control step, of displaying an image including at least the pixel of interest on display means such that 
images displayed on the present display apparatus and said one or more other display apparatuses form, as a whole, 
a complete enlarged image of the image corresponding to the input video signal. 

5 [0009] The present invention provides a first storage medium including a program stored thereon for causing a com- 
puter to control a display apparatus so as to display an image in accordance with an input video signal input from the 
outside, said program comprising: a classification step, of determining a class corresponding to a pixel of interest such 
that plural class reference pixels in the vicinity of the pixel of interest to be predicted are extracted from the input video 
signal and the class corresponding to the pixel of interest is determined from the extracted class reference pixels; a 

10 prediction coefficient generation step for generating a prediction coefficient corresponding to the class determined in 
the classification step; a pixel prediction step for predicting the pixel of interest such that plural prediction reference 
pixels in the vicinity of the pixel of interest are extracted from the input video signal, and the pixel of interest is predicted 
by means of a prediction operation using the extracted plural prediction reference pixels and the prediction coefficient; 
and a display control step, of displaying an image including at least the pixel of interest on display means such that 

15 images displayed on the present display apparatus and said one or more other display apparatuses form, as a whole, 
a complete enlarged image of the image corresponding to the input video signal. 

[0010] The present invention provides a first display system including at least a first display apparatus and a second 
display apparatus connected with each other, the first display apparatus comprising; display means for displaying an 
image; classifying means for determining a class corresponding to a pixel of interest such that plural class reference 

20 pixels in the vicinity of the pixel of interest to be predicted are extracted from the input video signal and the class 
corresponding to the pixel of interest is determined from the extracted class reference pixels; prediction coefficient 
generation means for generating a prediction coefficient corresponding to the class determined by the classifying 
means; pixel prediction means for predicting the pixel of interest such that plural prediction reference pixels in the 
vicinity of the pixel of interest are extracted from the input video signal, and the pixel of interest is predicted by means 

25 of a prediction operation using the extracted plural prediction reference pixels and the prediction coefficient; display 
control means for displaying an image including at least the pixel of interest such that images displayed on the present 
display apparatus and the second display apparatus form, as a whole, a complete enlarged image of the image cor- 
responding ito the input video signal; and transmission means for transmitting at least part of the predicted pixel of 
interest; the second display apparatus comprising: input means for inputting at least part of the predicted pixel of 

30 interest; and display means for displaying the enlarged image including at least the pixel of interest. 

[001 1] The present invention provides a second display apparatus, con nec table with one or more other display ap- 
paratuses and including display means for displaying an image, comprising: input means for inputting video signal 
output from one of other display apparatuses; image enlarging means for generating, from the input video signal, an 
enlarged image of the image corresponding to the input video signal; authentication means for performing mutual 

35 authentication with said one of other display apparatuses; and display control means for, if the authentication has been 
successfully passed, displaying an enlarged image generated by the image enlarging means on the display means 
such that images displayed on the display apparatus and said one or more other display apparatuses form, as a whole, 
a complete enlarged image. 

[0012] The present invention provides a second display method for a display apparatus, connectable with one or 
40 more other display apparatuses and including display means for displaying an image, to display an image, comprising: 
an input step, of inputting video signal output from one of other display apparatuses; an image enlarging step, of 
generating, from the input video signal, an enlarged image of the image corresponding to the input video signal; an 
authentication step, of performing mutual authentication with said one of other display apparatuses; and display control 
means for, if the authentication has been successfully passed, displaying an enlarged image generated by the image 
45 enlarging means on the display means such that images displayed on the display apparatus and said one or more 
other display apparatuses form, as a whole, a complete enlarged image. 

[0013] The present invention provides a second program for causing a computer to control a display apparatus 
connectable with one or more display apparatuses and including display means for displaying an image, said program 
comprising: an image enlarging step, of generating, from the input video signal, an enlarged image of the image cor- 

50 responding to the input video signal; an authentication step, of performing mutual authentication with said one of other 
display apparatuses; and a display control step of, if the authentication has been successfully passed, displaying an 
enlarged image generated by the image enlarging means on the display means such that images displayed on the 
display apparatus and said one or more other display apparatuses form, as a whole, a complete enlarged image. 
[0014] The present invention provides a second storage medium including a program stored thereon for causing a 

55 computer to control a display apparatus connectable with one or more other display apparatuses and including display 
means for displaying an image, said program comprising: an image enlarging step, of generating, from the input video 
signal, an enlarged image of the image corresponding to the input video signal; an authentication step, of performing 
mutual authentication with said one of other display apparatuses; and a display control step of, if the authentication 
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has been successfully passed, displaying an enlarged image generated by the image enlarging means on the display 
means such that images displayed on the display apparatus and said one or more other display apparatuses form, as 
a whole, a complete enlarged image. 

[0015] The present invention provides a second display system comprising at least a first display apparatus and a 
5 second display apparatus, the first display apparatus comprising: display means for displaying an image; and output 
means for outputting a video signal to be used by the second display apparatus to display an enlarged image, the 
second display apparatus comprising: input means for inputting the video signal output from the first display apparatus: 
image enlarging means for generating, from the input video signal, an enlarged image of the image corresponding to 
the input video signal; authentication means for performing mutual authentication with the first display apparatus; dis- 

10 play means for displaying an image; and display control means for, if the authentication has been successfully passed, 
displaying an enlarged image generated by the image enlarging means on the display means such that images dis- 
played on the first and second display apparatuses form, as a whole, a complete enlarged image. 
[0016] In the first display apparatus, display method, program, and storage medium, a prediction tap used to predict 
a pixel of interest selected from pixels constituting an image enlarged from an input image, and a class tap used to 

15 classify the pixel of interest into one of classes are extracted from the input image, and the pixel of interest is classified 
on the basis of the class tap. The pixel value of the pixel of interest is then predicted using the prediction tap and a tap 
coefficient which corresponds to the class of the pixel of interest and which is selected from tap coefficients which have 
been prepared by means of learning for each class. An enlarged image made up of predicted pixels is displayed on 
the display means so that images displayed on the present display apparatus and other display apparatus form, as a 

20 whole, a complete enlarged image. 

[0017] In the first display system, a prediction tap used to predict a pixel of interest selected from pixels constituting 
an image enlarged from an input image, and a class tap used to classify the pixel of interest into one of classes are 
extracted from the input image, and the pixel of interest is classified on the basis of the class tap. The pixel value of 
the pixel of interest is then predicted using the prediction tap and a tap coefficient which corresponds to the class of 

25 the pixel of interest and which is selected from tap coefficients which have been prepared by means of learning for 
each class. An enlarged image made up of predicted pixels is displayed on the display means so that the images 
displayed over the entire screen areas of the present display apparatus and other display apparatus form, as a whole, 
a complete enlarged image. 

[0018] In the second display apparatus, display method, program, and storage medium, an input image is converted 
30 into an enlarged image similar to the input image. If mutual authentication performed between the present display 
apparatus and one or more other display apparatuses has been successfully passed, enlarged images are displayed 
on the present display apparatus and the one or more other display apparatus so that the images displayed on the 
respective display apparatus form, as a whole, a complete enlarged image, 

[0019] In the second display system, an input image is converted into an enlarged image similar to the input image. 
35 if mutual authentication performed between the present display apparatus and one or more other display apparatuses 
has been successfully passed, enlarged images are displayed on the present display apparatus and the one or more 
other display apparatus, over the entire their screen areas, so that the images displayed on the respective display 
apparatus form, as a whole, a complete enlarged image. 

40 BRIEF DESCRIPTION OF THE DRAWINGS 

[0020] 

Figs. 1 A and 1 B are perspective views showing examples of constructions of a scalable TV system according to 
45 the present invention; 

Fig. 2 is a perspective view showing an example of the external structure of a master device; 

Figs. 3A to 3F are views, seen from different six sides, of the external structure of the master device; 

Fig. 4 is a perspective view showing an example of the external structure of a slave device; 

Figs. 5A to 5F are views, seen from different six sides, of the external structure of the slave device; 
50 Fig. 6 is a perspective view showing an example of an external structure of a dedicated rack for installing master 

device and slave devices of a scalable TV system; 

Fig. 7 is a plan view showing an example of an external structure of a remote commander 15; 
Fig. 8 is a plan view showing an example of an external structure of another remote commander; 
Fig. 9 is a plan view showing another example of an external structure of the remote commander; 
55 Fig. 10 is a block diagram showing an example of an electrical configuration of the master device; 

Fig. 11 is a block diagram showing an example of an electrical configuration of the slave device; 
Fig. 12 is a diagram showing the layer structure of the IEEE 1394 communication protocol; 
Fig. 13 is a diagram showing an address space according the CSR architecture; 
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Fig. 14 is a table showing offset addresses, names, and operations of a CSR; 
Fig. 15 is a diagram showing a general ROM format; 

Fig. 16 is a diagram showing details of bus__info_block, rooLdi rectory, and unit_directory; 
Fig. 17 is a diagram showing the structure of a PCR; 
5 Figs. 18A to 18D are diagrams showing structures of OMPR, oFCR, iMPR, and iPCR, respectively; 

Fig. 19 is a diagram showing a data structure of a packet of an AV/C command transmitted in an asynchronous 
transmission mode; 

Figs. 20A to 20C are diagrams showing specific examples of AV/C commands; 

Fig. 21 A and 21 B are diagrams showing specific examples of an AV/C command and a response thereto; 
10 Fig. 22 is a block diagram showing an example of a detailed structure of a signal processor; 

Fig. 23 is a flow chart showing a video data conversion performed by the signal processor 
Fig. 24 is a block diagram showing an example of the configuration of a learning apparatus; 
Fig. 25 is a diagram showing a process performed by a student data generator; 

Fig. 26 is a flow chart showing a learning process in terms of coefficient seed data, performed by the learning 
15 apparatus; 

Fig. 27 is a diagram showing a method of learning in terms of coefficient seed data; 

Fig. 28 is a block diagram showing another example of the configuration of the learning apparatus; 

Fig. 29 is a block diagram showing an example of the configuration of the signal processor; 

Fig. 30 is a flow chart showing a process performed by the master device; 
20 Fig. 31 is a flow chart showing an authentication process performed by the master device; 

Fig. 32 is a flow chart showing a process performed by the slave device; 

Fig. 33 is a flow chart showing an authentication process performed by the slave device; 

Fig. 34 is a flow chart showing a process, performed by the master device, on a closed caption; 
v Fig. 35 is a flow chart showing a process, performed by the slave device, on a closed caption; 
25 * Fig. 36 is a flow chart showing a partial enlarging process performed by the master device; 
\f Fig. 37 is a flow chart showing a partially enlarging process performed by the slave device; 
u Figs. 38A and 38 B are diagrams showing an example of a manner of displaying a partially enlarged image in a 
& scalable TV system; 

Fig. 39 is a flow chart showing a full image enlarging process performed by the master device: 
30 Fjgs. 40A and 40B are diagrams showing a method of determining a displaying area and an enlarging area; 

Fig. 41 is a flow chart showing a full-image enlarging process performed by the slave device; 

Figs. 42A to 42C are diagrams showing examples of manners of enlarging a full image in the scalable TV system; 

Fig. 43 is a flow chart showing an on-multiscreen displaying process performed by the master device; 

Fig. 44 is a flow chart showing a simultaneous control process performed by the master device; 
35 Figs. 45A and 45B are diagrams showing examples of images displayed in the scalable TV system by means of 

the simultaneous control process; 

Fig. 46 is a flow chart showing an individual device control process performed by the master device; 

Fig. 47 is a flow chart showing an individual device control process performed by the slave device; 

Fig. 48 is a flow chart showing a speaker control process performed by the master device; 
40 Fig. 49 shows an intensity-distance table; 

Fig. 50 is a diagram showing a method of calculating the distance to a remote commander; 

Fig. 51 is a flow chart showing a speaker control process performed by the slave device; 

Fig. 52 is a block diagram showing an example of the configuration of a speaker unit; 

Fig. 53 is a diagram showing an example of directivity; 
45 Fig. 54 is a diagram showing another example of directivity; 

Fig. 55 is a diagram showing a method of detecting the direction of the remote commander 

Fig. 56 is a diagram showing an example of the configuration of an I R receiver; 

Fig. 57 is a block diagram showing another example of an electrical configuration of the master device; 
Fig. 58 is a block diagram showing another example of an electrical configuration of the slave device; and 
50 Fig. 59 is a block diagram showing an example of a construction of a computer according to an embodiment of 

the present invention. 

DESCRIPTION OF THE PREFERRED EMBODIMENTS 

55 [0021] Fig. 1 is a perspective view showing an example of a construction of a scalable TV (Television) system (the 
term "system" is used in the present description to express a collection of a plurality of apparatuses logically coupled 
with each other, in which the apparatuses may or may not be located in a single case) according to the present invention. 
[0022] In an embodiment shown in Fig. 1 A, a scalable TV system includes nine television sets 1, 2 1V 2 12 , 2 13> 2 21 , 
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223, 2 31» 2 32 . and 233. In an embodiment shown in Fig. 1B, a scalable TV system includes twenty five television sets 

1 . 2 11. 2 12» 2 13- 2 14. 2 15* 2 2V 2 22« 2 23> 2 24. 2 25- 2 31» 2 32» 2 34» 2 35> 2 4L 2 42» 2 43' 2 44. 2 45l 2 51 , 2 52 , 2 53 , 254, and 255. 

[0023] Note that the number of television sets included in the scalable TV system is not limited to either 9 or 25. That 
is, the number of television sets included in the scalable TV system may be set to an arbitrary number equal to or 

5 greater than 2. The arrangement of television sets of the scalable TV system is not limited to either a 3 x 3 arrangement 
or a 5 x 5 arrangement shown in Fig. 1 A or 1 B. That is, television sets in the scalable TV system may be arranged in 
various fashions. For example, 1x2, 2X1, 2x3, or other arrangements may be employed. Furthermore, the positional 
arrangement of television sets in the scalable TV system is not limited to a matrix arrangement such as that shown in 
Fig. 1. For example, a pyramid-shaped arrangement may also be employed. 

10 [0024] In the scalable TV system, an arbitrary number of television sets may be arranged in horizontal and vertical 
directions. In this sense, the system is "scalable". 

[0025] The scalable TV system includes two types of television sets: a master device which can control other tele- 
vision sets; and slave devices which can be controlled by another television set but which cannot control any other 
television set 

75 [0026] In order that the scalable TV system has various capabilities which will be described later, it is required that 
the television sets in the scalable TV system should have a capability of operating as a member of the scalable TV 
system (hereinafter, a television set having such a capability will be referred to simply as a scalable device) and it is 
also required that at least one of members is a master device. In the embodiments shown in Fig. 1A and 1B, one of 
television sets of the scalable TV system (one located at the center, for example) is selected to be a master device 1 . 

20 [0027] As can be understood from the above description, if a system includes a television set which does not have 
the scalable capability, the system cannot operate as a scalable TV system. Furthermore, even when all television sets 
included in a system have the scalable capability, if they are all slave devices, the system cannot function as a scalable 
TV system. 

[0028] Therefore, to enjoy functions provided by a scalable TV system, a user has to purchase at least one or more 
25 master devices, or one master device and one or more slave devices. 

[0029] A master device can also operate as a slave device. Therefore, the scalable TV system can include a plurality 
of master devices. 

[0030] In the embodiment shown in Fig. 1 A, a master device 1 is located at the center (at a second place as counted 
from the left end and at a second place as counted from the top) of the 3 x 3 arrangement and the other eight television 
30 sets 2 n , 2 12 , 2 13 , 2 2n , 2^, 2 31 , 2 32 , and 233 are of the slave type. In the example shown in Fig. 1B, a scalable TV 
system includes 5x5 television sets, in which a television set 1 located at the center (at a third place as counted from 
the left end and at a third place as counted from the top) serves as a master device, and the other twenty four television 
sets 2 11f 2 12 , 2 13 , 2 14 , 2 15 , 2 21 , 2^, 2^, 2 24 , 2 25 , 2 31 , 2 32 , 234, 235, 2 41 , 2 42 , 2 43 , 2^, 2 45 , 2 51 , 2 52 , 253, 254, and 255 
serve as slave devices. 

35 [0031] Although in the example ehown in Fig. 1 , the master device 1 is placed at the center of the arrangement of 
television sets of the scalable TV system, the location of the master device 1 is not limited to the center of the arrange- 
ment of television sets, but the master device 1 may be placed at an arbitrary location such as the top left or bottom 
right location. 

[0032] In any case in which a master device 1 is located at an arbitrary place, a television set located as the center 
40 of the arrangement of the scalable TV system can be regarded as a master device in various processed described 
below. 

[0033] Hereinafter, for simplicity, the scalable TV system includes 3x3 television sets as shown in Fig. 1 A, and the 
master device 1 is assumed to be located at the center of the arrangement of television sets of the scalable TV system. 
[0034] The location of each slave device 2 in the scalable TV system is denoted by a suffix following "slave device 
45 2". For example, a slave device 2y denotes a slave device located in an \th row and a \th column (at an \th place as 
counted from the left end and at a jth place as counted from the top. 

[0035] Hereinafter, when it is not necessary to distinguish slave devices 2y from each other, a simple expression of 
"slave device(s) 2" will be used. 

[0036] Fig. 2 is a perspective view showing an example of a structure of a television set serving as a master device 1 . 
50 [0037] The television set used as the master device 1 has a display screen with a size of, for example, 14 inches or 
15 inches. The master device 1 includes a CRT (Cathode Ray Tube) 11 for displaying an image, located at the center 
of the front panel. Speaker units 12L and 12R for outputting a sound/voice are located on the left side and right side, 
respectively, of the front panel. 

[0038] An image is displayed on the CRT 11 in accordance with a television broadcasting signal received via an 
55 antenna (not shown). L (Left)-channel and R (Right)-channel voices/sounds associated with the image are output from 
speaker units 12L and 12R, respectively. 

[0039] A remote commander 15 for emitting an IR (Infrared Ray) is used in conjunction with the master device 1 . By 
operating the remote commander 15, a user can issue various commands such as a channel selection command, a 
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volume setting command, and the like to the master device 1 . 

[0040] The remote commander 15 is not limited to the one which communicates with the master device 1 via an 
infrared ray, but other types of wireless remote commanders such as that based on the BlueTooth (trademark) tech- 
nology may also be employed. 

5 [0041 ] The remote commander 1 5 can control not only the master device 1 but also slave devices 2. 

[0042] Fig. 3 shows an example of the structure of the master device 1 shown in Fig. 2, viewed from six different sides. 
[0043] That is, the structure of the master device 1 viewed from the front side is shown in Fig. 3A, the structure 
viewed from the upper side is shown in Fig. 3B, the structure viewed from the bottom side is shown in Fig. 3C, the 
structure viewed from the left side is shown in Fig. 3D, the structure viewed from the right side is shown in Fig. 3E, 

10 and the structure viewed from the back side is shown in Fig. 3F. 

[0044] Fixing mechanisms FIX-1 to FIX-4 are formed on the upper side (Fig. 3B), the bottom side (Fig. 3C), the left 
side (Fig. 3D), and the right side (Fig. 3E), respectively, of the master device 1. As will be described later, similar fixing 
mechanisms FlX-5 to FIX-8 are also formed on the upper side, the bottom side, the left side, and the right side of each 
television set serving as a slave device 2 so that when slave devices 2 or another master device 1 are placed on the 

15 upper side, below the bottom side, on the left side, or the on the right side of the master device 1 , the fixing mechanisms 
formed on the upper side, the bottom side, the left side, and the right side of the master device 1 fit with the fixing 
mechanisms with corresponding fixing mechanisms formed on the sides of the slave devices 2 or another master 
device 1 thereby ensuring that the master device 1 and the slave devices 2 or another master device are securely 
coupled with each other. This prevents the television sets in the scalable TV system from moving from their correct 

20 positions; 

[0045] Each fixing mechanism may be realized by means of a mechanical structure or another means such as a 
magnet. 

[0046] As shown in Fig. 3F, a terminal panel 21 , an antenna terminal 22, an input terminal 23, and an output terminal 
24 are disposed on the back side of the master device 1 . 
25 [0047] On the terminal panel 21, there are disposed eight IEEE (Institute of Electrical and Electronics Engineers) 
1394 terminals 21 1V 21 12 . 21 13 , 21 21 , 21 23. 21 31 , 21 32 , and 21 33 for electrical connection with eight slave devices 2 1V 
2 12' 2 13- 2 2V 2 23» 2 31» 2 32- and 2 33 ' n tne scalable TV system shown in Fig. 1A. 

[0048] In the example shown in Fig. 3F, in order to make it possible for the master device 1 to recognize the locations 
of the slave devices 2;j in the scalable TV system shown in Fig. 1 A, IKEE1 394 terminals 21 g connected to the respective 
30 slave devices 2§ are formed on the terminal panel 21 such that the locations of the IEEE1394 terminals 21jj on the 
terminal panel 21 correspond, when viewed from the back side, to the locations of the respective slave devices 2y in 
the scalable TV system shown in Fig. 1A. 

[0049] That is, in the example of the scalable TV system shown in Fig. 1 A, a user connects the master device 1 with 
the slave devices 2^ via the IEEE 1394 terminal 21 11f the slave device 2 12 via the IEEE 1394 terminal 21 12 , the slave 
35 device 2 13 via the IEEE1394 terminal 21 13 , the slave device 2 21 via the IEEE1394 terminal 21 21 , the slave device 2^ 
via the IEEE1394 terminal 21 23, the slave device 2 31 via the IEEE 1394 terminal 21 31 , the slave device 2 32 via the 
IEEE 1394 terminal 21 32> and the slave device 2^ via the IEEE 1394 terminal 21 33. 

[0050] In the scalable TV system shown in Fig. 1A. there is no specific limitation on which one of the IEEE1394 
terminals on the terminal panel 21 should be used to connect a slave device 2 t j. However, when a slave device 2 tJ is 
40 connected via a IEER1394 terminal other than the IEEE1394 terminal 21y, it is required to perform setting (by a user) 
so that the master device 1 can recognize that the slave device 2y is located in the ith row and jth column in the scalable 
TV system shown in Fig. 1A. 

[0051] Although in the example shown in Fig. 3F, the master device 1 is connected with eight slave devices 2^ to 
233 in a parallel fashion via eight IEEE1394 terminals 21^ to 21 33 formed on the terminal panel 21 , the master device 
45 1 may by connected with eight slave devices 2 in to 2 33 in a serial fashion. In this case, a slave device 2y is connected 
with the master device 1 via another slave device 2,y. However, also in this case, it is required to perform setting such 
the master device 1 can recognize that the slave device 2y is located in the ith row and \th column in the arrangement 
of the scalable TV system shown in Fig. 1A. Thus, the number of IEEE1394 terminals disposed on the terminal panel 
21 is not limited to 8. 

so [0052] Furthermore, the technique of the electrical connection between television sets in the scalable TV system is 
not limited to that based on the IEEE 1394 standard, but the electrical connection may also be accomplished using 
other techniques such as a LAN (according to the IEEE802 standard). Furthermore, in the electrical connection between 
television sets in the scalable TV system, wireless transmission may be employed instead of cable transmission. 
[0053] An antenna (not shown) is connected to the antenna terminal 22 via a cable so that a television broadcasting 

55 signal received by the antenna is supplied to the master device 1. The input terminal 23 is used to make connection 
with, for example, a VTR (Video Tape Recorder) to receive video data and audio data output from the VTR. Video data 
and audio data of, for example, a television broadcasting signal being received by the master device 1 are output from 
the output terminal 24. 
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[0054] Pig. 4 is a perspective view showing the structure of a television set serving as a slave device 2. 
[0055] The slave device 2 is a television set having the same screen size as that of the master device 1 shown in 
Fig. 2. The slave device 2 includes a CRT (Cathode Ray Tube) 31 for displaying an image, located at the center of the 
front panel. Speaker units 32L and 32R for outputting a sound/voice are located on the left side and right side, respec- 
5 tively, of the front panel. The screen size is not necessarily needed to be equal for the master device 1 and the slave 
devices 2. 

[0056] An image is displayed on a CRT 31 in accordance with a television broadcasting signal received via an antenna 
(not shown), and L-channel and R-channel audio signals associated with the image are output from speaker units 32L 
and 32 R, respectively. 

1 o [0057] There is also a remote commander 35, similar to that for use with the master device 1 , for emitting an infrared 
ray IR to control the slave device 2. A user can transmit various commands such as channel selection command or a 
volume control command to the slave device 2 by operating the remote commander 35. 
[0058] The remote commander 35 can control not only the slave device 2 but also the master device 1 . 
[0059] In order to realize the scalable TV system shown in Fig. 1A, a user has to purchase one master television 

15 system 1 and eight slave devices 2 11 to 233. If a remote commander 15 comes with the master device 1 and remote 
commanders 35 come with the respective eight slave devices 2 in to 233, the user will have nine remote commanders, 
which will cause the user to have to make a troublesome job to manage the remote commanders. 
[0060] To avoid the above problem, the remote commander 35 of each slave device 2 may be sold as an optional 
part separately from the slave device 2. Similarly, the remote commander 15 of the master device 1 may be sold as 

20 an optional part separately from the master device 1 . 

[0061] Because both remote commanders 15 and 35 are capable of controlling the master devices 1 and the slave 
devices 2, the user can control the master device 1 and any slave device 2 using a single remote commander 15 or 35. 
[0062] Fig. 5 shows an example of the structure of the slave device 2 shown in Fig. 4, viewed from six different sides. 
[0063] That is, the structure of the slave device 2 viewed from the front side is shown in Fig. 5A, the structure viewed 

25 from the upper side is shown in Fig. 5B, the structure viewed from the bottom side is shown in Fig. 5C, the structure 
viewed from the left side is shown in Fig. 5D, the structure viewed from the right side is shown in Fig. 5E, and the 
structure viewed from the back side is shown in Fig. 5F. 

[0064] Fixing mechanisms FIX-5 to FIX-8 are formed on the upper side (Fig. 5B), the bottom side (Fig. 5C), the left 
side (Fig. 5D), and the right side (Fig. 5E), respectively, of the slave device 2 so that when the master device 1 or other 

30 slave devices are placed on the upper side, below the bottom side, on the left side, or the on the right side of the slave 
device 2, the fixing mechanisms formed on the upper side, the bottom side, the left side, and the right side of the slave 
device 2 fit with the fixing mechanisms with corresponding fixing mechanisms formed on the sides of the master device 
1 or other slave devices thereby ensuring that the slave device 2 and other slave devices 2 or the master device 1 are 
seourely coupled with each other. 

35 [0065] As shown in Fig. 5F, a terminal panel 4 1 , an antenna terminal 42, an input terminal 43, and an output terminal 
44 are disposed on the back side of the slave device 2. 

[0066] On the terminal panel 41, there is disposed an IEEE1394 terminal 4^ for electrically connecting the slave 
device 2 with the master device 1. In the case in which the slave device 2 is used, for example, as a slave device 2 ni 
placed at the upper left location in the arrangement of the scalable TV system shown in Fig. 1A, the IEEE1394 terminal 
40 41 n on the terminal panel 41 is connected to the IEEE1394 terminal 21 ni on the terminal panel 21 shown in Fig. 3F 
via an IEEE1394 cable (not shown). 

[0067] The number of IEEE 1394 terminals on the terminal panel 41 is not limited to 1 . 

[0068] An antenna (not shown) is connected to the antenna terminal 42 via a cable (not shown) so that a television 
broadcast signal received by the antenna is applied to the slave device 2. The input terminal 43 is used to make 
45 connection with, for example, a VTR so as to receive video data and audio data output from the VTR. Video data and 
audio data of, for example, a television broadcast signal being received by the slave device 2 are output from the output 
terminal 44. 

[0069] As described above, the scalable TV system shown in Fig. 1 A is constructed by placing a total of nine television 
sets including one master device 1 and eight slave devices Z t1 to 233 in a 3 x 3 array fashion. 

50 [0070] Although in the above-described example, the scalable TV system shown in Fig. 1 A is constructed by placing 
television sets serving as a master device or slave devices side by side in the horizontal and vertical directions such 
that adjacent television sets are directly connected with each other without being spaced, television sets may also be 
placed on a rack such as that shown in Fig. 6 designed for use in the scalable TV system. Use of such a rack designed 
for use in the scalable TV system makes it possible to prevent the television sets in the scalable TV system from moving 

55 from their correct positions in a more secure fashion. 

[0071] In the case in which the scalable TV system is constructed by placing television sets serving as a master 
device or slave devices side by side in the horizontal and vertical directions such that they are directly connected with 
each other without being spaced, it is impossible to place, for example, the master device 1 in the second row and in 
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the second column as shown in Fig. 1 A unless there is at least a slave device 2 32 - In contrast, in the case in which the 
rack, such as that shown in Fig. 6, designed for use in the scalable TV system is used, the master device 1 can be 
placed in the second row and in the second column even when there is no slave device 2 32 placed in the third row in 
the second column. 

5 [0072] Fig. 7 is a plan view showing an example of the structure of the remote commander. 

[0073] A select button switch 51 accepts operations in an upward direction, a downward direction, a leftward direction, 
and a rightward diction, and also in four slanting directions between adjacent two directions of the former four directions. 
The select button switch 51 also accepts an operation (selection) performed in a direction (select direction) perpen- 
dicular to the upper surface of the remote commander 15. If a menu button switch 54 is pressed, a menu screen is 

10 displayed on the CRT 11 of the master device 1 (or the CRT 31 of the slave device 2) thereby allowing a user to perform 
various kinds of setting (such as specifying of the location of a particular slave device in the arrangement of the scalable 
TV.system) or input commands to request various kinds of processing. 

[0074] When the menu screen is displayed, a cursor for pointing to a particular item in the menu is also displayed 
on the CRT 1 1 . The cursor can be moved by operating the select button switch 51 . More specifically, the cursor moves 

15 in a direction corresponding to a direction in which the select button switch 51 is operated. When the cursor is on a 
particular item, if the select button switch 51 is operated in the select direction, the item pointed to by the cursor is 
selected. In the present embodiment, as will be described in further detail later, items displayed in the menu include 
icons. A desired icon can be clicked by operating the select button switch 51 in the select direction. 
[0075] An exit button switch 55 is used to exit the menu screen to return to an original normal screen. 

20 [0076] Volume button switches 52 are used to increase or decrease the sound volume. Channel up/down button 
switches 53 are used to increase or decrease the channel number of a broadcast channel to be received. 
[0077] If one of numerical button switches (ten-key switches) 58 labeled numerals 0 to 9 is pressed, a numeral 
labeled on the pressed numerical button switch is input. If an enter button switch 57 is pressed after completion of 
inputting one or more numerals using numerical button switches 58, a command indicating the end of inputting of 

25 numerals is input. When the channel is switched, a new channel number or the like is displayed in the OSD (On Screen 
Display) fashion on the CRT 1 1 of the master device 1 (or the CRT 3 1 of the slave device 2) for a predetermined period 
of time. A display button 56 is used to turn on/off the displaying of the channel number being currently selected or the 
volume level being currently selected. 

[0078] A TV/video button switch 59 is used to switch the input applied to the master device 1 (or the slave device 2) 

30 between the input given by a tuner 121 which is disposed in the master device 1 and which will be described later with 
reference to Fig 10 (or the input given by a tuner 141 which will be described later with reference to Fig. 11) and the 
input given via the input terminal 23 shown in Fig. 3 (or the input terminal 43 shown in Fig. 5). A TV/DSS button switch 
60 is used to switch the reception mode between a TV mode in which ground wave broadcast is received via the tuner 
121 and a DSS (Digital Satellite System (trademark of Hughes Communications, Inc.) mode in which satellite broadcast 

35 is received. If the channel is switched by operating one or more numerical button switches 58, data indicating the 
previous channel is retained. If a jump button switch 61 is pressed, the channel is switched to the previous channel. 
[0079] A language button 62 is used to select a desired language when two or more languages are available in the 
broadcast being received, When video data being displayed on the CRT 11 includes closed caption data, if a guide 
button switch 63 is operated, the closed caption data is displayed. A favorite button switch 64 is used to select a favorite 

40 channel which has been selected by a user in advance. 

[0080] A cable button switch 65, a TV switch 66, and a DSS button switch 67 are used to select a device category 
to be controlled by command codes transmitted via an infrared ray emitted from the remote commander 15. That is, 
the remote commander 15 (and also the remote commander 35) is capable of remotely controlling not only television 
sets serving as the master device 1 or slave devices 2 but also an STB (Set Top Box) or an IRD (Integrated Receiver 

45 and Decoder) (not shown), and the cable button switch 65, the TV switch 66, and the DSS button switch 67 are used 
to select a device to be controlled. For example, if the cable button switch 65 is pressed, the STB for receiving a signal 
via a CATV network is selected as a device to be controlled by the remote commander. In the sate in which the STB 
is selected, if the remote commander 15 is operated, an infrared array carrying one of command codes associated 
with the STB is emitted from the remote commander 15. Similarly, if the TV button switch 66 is pressed, the master 

50 device 1 (or the slave device 2) is selected as a device to be controlled by the remote commander 15. The DSS button 
switch 67 is used to seiect the IRD for receiving a signal transmitted from a satellite as a device to be controlled by the 
remote commander 1 5. 

[0081] LEDs (Light Emitting Diodes) 68, 69, and 70 are lit when the cable button switch 65, the TV button switch 66, 
or the DSS button switch 67 is pressed so that a user can know which device category is currently selected as a device 
55 category to be controlled by the remote commander 15. The LEDs (Light Emitting Diodes) 68, 69, and 70 are turned 
off when the cable button switch 65, the TV button switch 66, or the DSS button switch 67 is turned off. 
[0082] A cable power button switch 71, a TV power button switch 72, and a DSS power button switch 73 are used 
to turn on/off the power of the STB, the master device 1 (or the slave device 2), or the IRD. 
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[0083] A muting button switch 74 is used to set or release the master device 1 (or the slave device 2) into or from a 
muted state. A sleep button switch 75 is used to set or reset the sleep mode in which electric power is automatically 
turned off at a specified time or when a specified period of time has elapsed. 

[0084] An infrared ray emitter 76 emits an infrared ray in response to an operation performed on the remote com- 
5 mander 15. 

[0085] Fig. 8 is a plan view showing an example of the structure of the remote commander 35 for use with the slave 
device 2. 

[0086] The remote commander 35 is made up of parts such as a select button switch 81 and an infrared ray 

emitter 106 similar to those such as the select button switch 51,..., and the infrared ray emitter 76 of the remote com- 
10 mander 15 shown in Fig. 7, and thus further description is not provided herein. 

[0087] Fig. 9 is a plan view showing another example of the structure of the remote commander 1 5 used to control 
the master device 1. 

[0088] In the example shown in Fig. 9, instead of the select button switch 51, shown in Fig. 7, capable of being 
operated in eight directions, there are provided four arrow buttons 111 , 112, 113, and 114 pointed in up, down, left, and 

15 right directions, respectively, and a select button switch 110. Furthermore, in the example shown in Fig. 9, a cable 
button switch 65, a TV button switch 66, and a DSS button switch 67 are of the self-lit type, and thus the LEDs 68 to 
70 employed in the example ehown in Fig. 7 are not provided. LEDs (not shown) are placed on the rear side of the 
restive button switches 65 to 67 so that when one of button switches 65 to 67 is pressed, an LED corresponding to the 
pressed button switch is turned on or off. 

20 [0089] The other buttons are substantially similar to those shown in Fig. 7, although their locations are different. 

[0090] The remote commander 35 used to control the slave device 2 may also be constructed in a similar manner 
to that shown in Fig. 9. 

[0091] The remote commander 15 may include a gyroscope for detecting the movement of the remote commander 
1 5. This makes it possible for the remote commander 15 to detect the moving direction and the moving distance of the 

25 remote commander 15, using the gyroscope disposed in the remote commander 15, and move the cursor displayed 
on the menu screen in a direction by a distance corresponding to the detected direction and distance. In the case in 
which the remote commander 15 includes such a gyroscope, it becomes unnecessary for the select button switch 51 
in the example shown in Fig. 7 to have the capability of detecting the eight directions in which the select button switch 
51 is operated, while it becomes unnecessary for the example shown in Fig. 9 to include the arrow button switches 

30 111 to 114. The remote commander 35 may also include a similar gyroscope. 

[0092] Fig. 1 0 shows an example of an electrical configuration of the master device 1 . 

[0093] A television broadcasting signal received by the antenna (not shown) is applied to a tuner 121 . The tuner 121 
detects and demodulates the television broadcasting signal under the control of a CPU 129, The output of the tuner 
121 is applied to a QPSK (Quadrature Phase Shift Keying) demodulator 122. The QPSK demodulator 122 QPSK- 
35 demodulates the applied signal under the control of the CPU 129 and outputs the resultant QPSK-demodulated signal 
to an error correction circuit 123. The error correction circuit 123 detects and corrects an error under the control of the 
CPU 129 and outputs the resultant corrected signal to a demultiplexer 124. 

[0094] Under the control of the CPU 129, the demultiplexer 124 descrambles, if required, the signal received from 
the error correction circuit 123 and then extracts TS (Transport Stream) packets of a particular channel. The demulti- 
40 plexer 1 24 supplies TS packets associated with video data to an MPEG (Moving Picture Experts Group) video decoder 

125 and also supplies TS packets associated with audio data to an MPEG audio decoder 126. The demultiplexer 124 
supplies TS packets included in the output of the error correction circuit 123 to the CPU 129, as required. The demul- 
tiplexer 124 also receives video data or audio data (which may be in the form of TS packets) from the CPU 129 and 
supplies the received video data or audio data to the MPEG video decoder 125 or the MPEG audio decoder 126. 

45 [0095] The MPEG video decoder 125 performs MPEG-decoding on the video data in the form of TS packets received 
from the demultiplexer 124 and supplies the resultant decoded data to a frame memory 127. The MPEG audio decoder 

126 performs MPEG-decoding on the audio data in the form of TS packets received from the demultiplexer 124. L- 
channel audio data and R-channel audio data obtained as a result of decoding performed by the MPEG audio decoder 
126 are supplied to the speaker units 12L and 12R, respectively. 

50 [0096] The frame memory 127 temporarily stores the video data received from the MEPG video decoder 125. After 
temporarily storage, the frame memory 127 outputs the video data to an NTSC (National Television System Committee) 
encoder 128. The NTSC encoder 128 converts the video data received from the frame memory 127 into video data in 
the NTSC format and the outputs the resultant NTSC video data to the CRT 11. The CRT 11 displays an image in 
accordance with the received video data. 

55 [0097] The CPU 129 performs various processes in accordance with programs stored in an EEPROM (Electrically 
Erasable Programmable Read Only Memory) 130 or a ROM (Read Only Memory) 131 to control the tuner 121, the 
QPSK demodulator 122, the error correction circuit 123, the demultiplexer 124, the IE EE 1394 interface 133, the modem 
136, the signal processor 137, and the unit driver 138. The CPU 129 supplies the data received from the demultiplexer 
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124 to the IEEE1394 interface 133, and the data received from the IEEE1394 interface 133 to the demultiplexer 124 
or the signal processor 137. Furthermore, the CPU 129 performs a process in response to a command received from 
the front panel 134 or the IR receiver 135. Furthermore, the CPU 129 controls the modem 136 to access a server (not 
shown) via a telephone line and acquires an updated program or necessary data. 
5 [0098] The EEPROM 1 30 is used to store data or a program which is necessary to be retained even after electrical 
power is turned off. The ROM 131 stores a program such as an IPL (Initial Program Loader). The data or the program 
stored in the EEPROM 130 can be updated by means of overwriting. 

[0099] The RAM 1 32 is used to temporarily store a program or data which is necessary in the operation performed 
by the CPU 129. 

10 [0100] The IEEE1394 interface 133 serves as a communication interface according to the IEEE1394 standard and 
is connected with the terminal panel 21 (more specifically, with the IEEE1394 terminals 21 in to 21 33 of the terminal 
panel 21). That is, the IEEE1394 interface 133 transmits data supplied from the CPU 129 to the outside in accordance 
with the IEEE1 394 standard and transfers data transmitted from the outside in accordance with the IEEE1 394 standard 
to the CPU 129. An external device can be connected to the terminal panel 21 via an IEEE1394 cable 21a. 

15 [01 01] The front panel 1 34 is disposed in a partial area of the front surface of the master device 1 , although it is not 
shown in Fig. 2 or 3. On the front panel 134, buttons switches similar to some of buttons switches of the remote 
commander 15 (Fig. 7 or 9). If one of button switches on the front panel 134 is operated, a command corresponding 
to the operation performed on the button is supplied to the CPU 129. In response, the CPU 129 performs an operation 
in accordance with the operation signal received from the front panel 134. 

20 [0102] The* IR receiver 135 receives an infrared ray transmitted from the remote commander 15 in response to an 
operation performed on the remote commander 15. The IR receiver 135 converts the received infrared ray into an 
electrical signal and supplies the resultant electrical signal to the CPU 129. In response, the CPU 129 performs a 
process in accordance with the signal received from the IR receiver 135. That is, the CPU 129 performs a process 
corresponding to the operation performed on the remote commander 1 5. 

25 [0103] The modem 136 controls the communication performed via the telephone line such that data supplied from 
the CPU 129 is transmitted over the telephone line and such that data received via the telephone line is transferred to 
the CPU 129. 

[0104] The signal processor 137 includes a DSP (Digital Signal Processor 137A, an EEPROM 137B, and a RAM 
137C, and performs various kinds of digital signal processing on video data stored in the frame memory 127, under 

30 the control of the CPU 1 29. 

[01 05] More specifically, the DSP 1 37 A performs various kinds of signal processing using data stored in the EEPROM 
137B as required, in accordance with a program stored in the EEPROM 137B. The EEPROM 137B stores a program 
and/or data used by the DSP137A in performing various processes. The RAM 137C is used to temporarily store a 
program and/or used by the DSP137A in performing various processes. 

35 [0106] The data or the program stored in the EEPROM 137B can be updated by means of overwriting. 

[01 07] The signal processing performed by the signal processor 137 includes, for example, decoding of closed cap- 
tion data, superimposing of closed caption data onto video data stored in the frame memory 127, scaling of video data 
stored in the frame memory 127, and removing of noise. The signal processor 137 also generates OSD data to be 
OSD-displayed and superimposes it onto video data stored in the frame memory 127. 

40 [0108] The unit driver 138 drives, under the control of the CPU 129, the speaker units 12L and 12R so that the 
principal axis of the directivity of the speaker system including the speaker units 12L and 12R is directed in a desired 
direction. 

[01 09] In the master device 1 constructed in the above-described manner, an image and a sound/voice associated 
with a television broadcasting program are output as described below 

45 [01 1 0] That is, a television broadcasting signal in the form of a transport stream received by the antenna is supplied 
to the demultiplexer 124 via the tuner 121, the QPSK demodulator 122, and the error correction circuit 123. The de- 
multiplexer 124 extracts TS packets of a program from the transport stream and supplies TS packets of video data 
and audio data to the MPEG video decoder 125 and the MPEG audio decoder 126, respectively. 
[01 1 1] The MPEG video decoder 125 performs MPEG-decoding on the TS packets received from the demultiplexer 

50 124. The video data obtained as the result of the MPEG-decoding is supplied from the MPEG video decoder 125 to 
the CRT 11 via the frame memory 127 and the NTSC encoder 128. 

[01 12] On the other hand, the MPEG audio decoder 126 performs MPEG-decoding on the TS packets received from 
the demultiplexer 124 and the audio data obtained as the result of the MPEG-decoding is supplied from the MPEG 
video decoder 126 to the speaker units 12L and 12R. 
55 [0113] Fig. 1 1 shows an example of an electrical configuration of a slave device 2. 

[01 14] The slave device 2 is made up of parts such as a tuner 141 and a unit driver 158 similar to the tuner 1 21 

and the unit driver 138 shown in Fig. 10, and thus a further description is not provided herein. 

[0115] The master device 1 and the slave device 2 both have their own antenna terminals 22 and 42 as shown in 
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Figs. 3F and 5F. Therefore, the antenna can be connected (via cables) to the master device 1 and the slave devices 
2 of the scalable TV system shown in Fig. 1. However, if the antenna is connected to the master device 1 and all slave 
devices 2, the connection becomes complicated. In the present scalable TV system, to avoid such complexity, the 
antenna may be connected to only one of the television sets of the scalable TV system, and a television broadcasting 

5 signal received by that television set may be distributed to the other television sets by means of IEEE 1 394 transmission. 
[01 16] In the present embodiment, the IEEE 1394 terminal 21 y (Fig. 3) on the terminal panel 21 of the master device 
1 and the IEEE1394 terminal 411 (Fig. 5) on the terminal panel 41 of the slave device 2y are connected to each other 
via an IEEE 1394 cable thereby electrically connecting the master device 1 and the slave device 2 to each other so as 
to allow the master device 1 and the slave device 2 to transmit various data to each other by means of IEEE 1394 

10 transmission (according to the IEEE 1394 standard). 

[0117] The IEEE1394 transmission process is described below with reference to Figs. 12 to 21. 
[0118] The IEEE1394 standard is one of standards for serial buses, According to the IEEE1394 standard, data is 
allowed to be transmitted isochronously, and thus this technology is suitable for transmission of data such as image 
data or audio data which is necessary to be played back in real time. 

15 [01 19] The IEEE1394 transmission allows data to be transferred isochronously at intervals of 125 \is using an up to 
125 us transmission band (called so although it is actually time). Within the transmission band described above, a 
plurality of channels can be used for isochronous transmission. 
[0120] Fig. 12 shows the layer structure of the IEEE1394 communication protocol. 

[0121] The IEEE1394 communication protocol has a 3-layer structure including a transaction layer, a link layer, and 
20 a physical layer. The respective layers communicate with each other and also with serial bus management. The trans- 
action layer and the link layer also communicate with an application at a higher level. In the communication, 4 types 
of messages are transmitted (received). They are request, indication, response, and confirmation messages. In Fig. 
12, arrows denote messages in communication. 

[0122] In Fig. 12, each arrow is labeled a message name. Message names with a suffix " req" denote request mes- 

25 sages, and message names with a suffix ".ind" denote indication messages. Message names with a suffix ".resp" 
denote response messages, and message names with a suffix ".conf denote confirmation messages. For example, 
TR_COHT.req is a request message transmitted from the serial bus manager to the transaction layer. 
[01 23] In response to a request from an application, the transaction layer provides asynchronous transmission serv- 
ice to allow data communication with another IEEE1394 device (having an IEEE1394 interface) on the basis of the 

30 request/response protocol according to the ISO/IEC13213 standard. In data transmission schemes according to the 
IEEE1 394 standard, in addition to the isochronous transmission, asynchronous transmission is also allowed, and asyn- 
chronous transmission is dealt with in the transaction layer. In asynchronous transmission, data is transmitted between 
IEEE1394 devices via three transactions which are units processed by the transaction layer: a read transaction; a write 
transaction; and a lock transaction. 

35 [0124] The link layer provides data transmission service using an acknowledge message and performs address 
processing, data error detection, and data framing. Transmission of a packet performed by the link layer is called a 
subaction. There are two types of subactions: asynchronous subactions and isochronous subactions. 
[0125] In asynchronous subactions, data it transmitted to a specified address in a node (unit accessible in IEEE 1394) 
identified by physical ID (identification). In response to receiving the data, the node returns an acknowledge message. 

40 However, in the case of asynchronous broadcast subactions, in which data is transmitted to all nodes on an IEEE1394 
serial bus, nodes do not return an acknowledge message in response to receiving the data. 

[0126] On the other hand, in isochronous subactions, data is transmitted at fixed intervals (of 125 us) to a specified 

channel number. In the case of isochronous subactions, no acknowledge message is returned. 

[0127] The physical layer converts logical symbols used by the link layer into electrical signals. Furthermore, the 

45 physical layer performs processing in response to an arbitration request issued by the link layer (when there are two 
or more nodes which are requesting IEEE1394 communication). When the IEEE1394 serial bus is reset, the physical 
layer performs reconfiguration of the IEEE1394 serial bus and automatically performs physical ID assignment 
[01 28] In the management of the serial bus, basic bus control capabilities are achieved in accordance with the ISO/ 
I EC 132 12 CSR (Control and Status Register) architecture. More specifically, the capabilities of the serial bus manage- 

50 ment include a node controller, an isochronous resource manager, and a bus manager. The node controller controls 
the status and physical ID of each node and also controls the transaction layer, the link layer, and the physical layer 
The isochronous resource manager provides information about availability of resources used in isochronous commu- 
nication. To perform isochronous communication, it is required that devices connected to the IEEE1394 serial bus 
include at least one IEEE 1394 device having the isochronous resource manager. The bus manager performs optimi- 

55 zation of use of the IEEE1394 serial bus, which is the highest-level capability of those provided by the serial bus 
management The isochronous resource manager and the bus manager may or may not exit 

[0129] In connection among IEEE1394 devices, both a branching-at-node connection and a node daisy chain con- 
nection are allowed. However, if a new IEEE 1394 device is connected, bus resetting is performed to detect a tree 
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structure and determine a root node, physical IDs, an isochronous resource manager, a cycle master, and a bus man- 
ager. 

[0130] In the detection of the tree structure, parent-child relationships among IEEE1394 devices are determined. 
The root node specifies a node permitted, via arbitration, to use the IEEE1 394 serial bus. Physical IDs are determined 
5 by transmitting packets called self-ID packets to the respective nodes. Each self-ID packet transmitted to a node in- 
cludes information indicating the data transmission rate of that node and information indicating whether the node can 
act as an isochronous resource manager. 

[0131] The isochronous resource manager is, as described earlier, a node which provides information about the 
status of availability of resources used in isochronous communication. The isochronous resource manager includes a 
10 BAND Wl DTH_AVAI LABLE register and a CHANNELS. AVAILABLE register, which will be described later. The iso- 
chronous resource manager also includes a register for storing data indicating the physical ID of a node serving as 
the bus manager. In a case in which there is no bus manager in IEEE1 394 devices connected as nodes to the IEEE1 394 
serial bus, the isochronous resource manager also serves as a simplified bus manager. 

[0132] The cycle master transmits a cycle start packet over the IEEE1394 serial bus at isochronous transmission 
15 intervals of 125 us. To this end, the cycle master includes a CYCLE__TIME register serving as a cycle time counter to 

determine the transmission timing at intervals of 125 us. The root node serves as the cycle master. However, when 

the root node does not have the capability of cycle master, the bus manger changes the root node. 

[0133] The bus manager manages the power of the IEEE1 394 serial bus and changes the root node if required. 

[0134] If the bus is reset and if further setting associated with the isochronous manage is performed, it becomes 
20 possible to perform data transmission via the IEEE 1394 serial bus. 

[0135] In isochronous transmission, which is one of data transmission schemes according to the IEEE1 394 standard, 

a transmission band and a transmission channel are first assigned, and then data is transmitted in the form of packets 

(isochronous packets). 

[0136] That is, in isochronous transmission, the cycle master first broadcasts a cycle start packet at intervals of 125 
25 us over the IEEE1394 serial bus. If the cycle start packet has been broadcasted, it becomes possible to transmit 
isochronous packets. 

[01 37] . To perform isochronous transmission, it is required to declare the use of resource for isochronous transmission 
by rewriting the BANDWIDTH. AVAILABLE register to assign a transmission bandwidth provided by the isochronous 
resource manager and rewriting the CHANNELS. AVAILABLE register to assign a channel. 
30 [0138] Each of the BANDWIDTH. AVAILABLE register and the CHANNELS. AVAILABLE register is assigned as a 
CSR (Control and Status Register) having a 64-bit address space according to the ISO/1EC13213 standard (the CSR 
will be described in further detail later). 

[0139] The BANDWIDTH.AVAILABLE register is a register for storing 32-bit data whose lower-order 13 bits is used 
to indicate a currently available transmission bandwidth (bw.remaining). 

35 [0140] The BANDWI DTH_ AVAI LABLE register is initially set to 00000000000000000001001100110011B (where B 
denotes that the value preceding B is represented in a binary notation) (= 491 5), for the following reason. In IEEE1394, 
the unit of time is defined as a time needed to transmit 32 bits at 1572.864 Mbps (bit per second). Therefore, 125 us 
corresponds to 00000000000000000001100000000000B (= 6144). However, in IEEE1394, the bandwidth available 
for isochronous transmission is 80% of one cycle period of 125 us, and thus the maximum bandwidth available for 

40 isochronous transmission is 100 us. Therefore, the BANDWIDTH.AVAILABLE register is initially set to 
00000000000000000001001100110011B (- 4915). 

[01 41 ] The rest of the bandwidth, that is 25 us remaining after 1 00 us of 125 us is used for isochronous transmission, 
is used for asynchronous transmission, which is performed to read data stored in the BANDWIDTH.AVAILABLE register 
or the CHANNELS. AVAILABLE register. 

45 [0142] To start isochronous transmission, it is required that a transmission bandwidth for use in the isochronous 
transmission has been made available. For example, in a case in which a transmission bandwidth of 10 us in the total 
bandwidth of 125 us is used for isochronous transmission, it is required that a transmission bandwidth of 10 us be 
assigned for isochronous transmission. The assignment of the transmission bandwidth is performed by rewriting the 
value stored in the BANDWIDTH.AVAILABLE register. More specifically, in the case in which a bandwidth of 10 us is 

50 assigned for isochronous transmission, 492 corresponding to 10 us is subtracted from the value stored in the 
BANDWIDTH.AVAILABLE register and the resultant value is set into the BANDWIDTH.AVAILABLE register. For ex- 
ample, when the current value of the BANDWIDTH.AVAILABLE register is 4915 (in the case in which isochronous 
transmission is not performed at all), a bandwidth of 10 us is assigned for isochronous transmission by rewriting the 
current value of 491 5 of the BANDWIDTH.AVAILABLE register into 4423 

55 (00000000000000000001000101 000111B) which is obtained by subtracting 492 corresponding to 10 us from 4915. 
[0143] If the result of subtraction of a transmission bandwidth to be assigned (used) from the current value of the 
BANDWIDTH.AVAILABLE register becomes smaller than 0, the transmission bandwidth cannot be assigned, and thus 
the BANDWIDTH.AVAILABLE register is not rewritten. In this case, isochronous transmission cannot be performed. 
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[0144] To perform isochronous transmission, it is also required to assign a transmission channel, in addition to a 
transmission bandwidth. The assignment of a transmission channel is performed by rewriting the 
CHANNELS. AVAILABLE register. 

[0145] The CHANNELS_AVAILABLE register ia a 64-bit register, each bit of which corresponds to a channel. When 
5 an nth bit (as counted from the least significant bit) is equal to 1 , an (n-1)f/? channel is not used, while when the nth bit 
is equal to 0, the (n-1 )th channel is being used. Therefore, when any channel is not used, the CHANNELS, AVAILABLE 
register has a value 

111111111111111111111111111111111111111111111111111111111111 1111B. For example, when a first channel is as- 
signed, the CHANNELS_AVAILABLE register is rewritten into 
10 111111111111111111111111111111111111111111111111111111111111 1101B. 

[0146] Because the CHANNELS. AVAILABLE register has a storage capacity of 64 bits as described earlier, it is 
possible to assign 64 channels from 0th to 63rd channels. Note that the 63rd channel is a special channel used to 
broadcast an isochronous packet 

[01 47] Because isochronous transmission is performed after assigning a transmission bandwidth and a transmission 
15 channel as described above, the transmission rate in the isochronous transmission can be guaranteed. Therefore, 
isochronous transmission is suitable in particular for transmission of data such as video data or audio data needed to 
play back in real time. 

[0148] As described above, IEEE 1394 transmission is based on the CSR architecture using a 64-bit address space 
according to the ISO/IEC13213 standard. 

20 [0149] Fig. 13 shows an address space of based on the CSR architecture. 

[01 50] High-order 16 bits of the CSR are used to represent a node ID of a node, and the remaining 48 bits are used 
to specify an address space assigned to the node. The high-order 16 bits are divided into a 1 0-bit part indicating a bus 
ID and a 6-bit part indicating a physical ID (node ID in a narrow sense). A value whose all bits are equal to 1 is used 
for a special purpose, and thus 1 023 buses and 63 nodes can be specified. 

25 [0151] In the 256-terabytes address space defined by lower-order 48 bits of the CSR, a space defined by higher- 
order 20 bits is divided into spaces including an initial register space used by a 2048- byte CSR register or an IEEE 1394 
register, a private space, and an initial memory space. In the case in which the space defined by the high-order 20 bits 
is used as the initial register space, the space defined by the lower-order 28 bits is used as a configuration ROM, an 
initial unit space used for a purpose specific to a node, or plug control registers (Peas). 

30 [0152] Fig. 14 shows offset addresses, names, and functions of main CSRs. 

[01 53] In Fig. 14, "offset" fields are used to describe offset addresses with respect to an address of FFFFFOOOOOOOh 
(h denotes that a value preceding h is represented in a hexadecimal notation) from which the initial register space 
begins. As for the BANDWIDTH. AVAILABLE register at an offset address of 220h, used to indicate the bandwidth 
assignable to isochronous communication as described earlier, only the value stored in the BAND WIDTH. AVAILABLE 

35 register of a node serving as the isochronous resource manager is valid. That is, although CSRs shown in Fig. 13 are 
possessed by each node, only the BANDWIDTH. AVAILABLE register possessed by the isochronous resource man- 
ager is valid. This means that, in affect, the BANDWIDTH.AVAILABLE register is possessed only by the isochronous 
resource manager. 

[01 54] The bits of the CHANNELS.AVAI LABLE register at offset addresses of 224h to 228h correspond to respective 
40 channel numbers from 0 to 63, as described earlier. When a particular bit is equal to 0, a corresponding channel is 
already assigned. Also in the case of the CHANNELS.AVAI LABLE register, only the CHANNELS. AVAILABLE register 
of a node serving as the isochronous resource manager is valid. 

[0155] Referring again to Fig. 13, in accordance with the general ROM format, a configuration ROM is placed at 
addresses of 400h to 800h in the initial register space. 

45 [01 56] Fig. 1 5 shows the general ROM format. 

[0157] Nodes, which are units accessible on the IEEE 1394 serial bus, may include plural units which use in common 
the same address space but operate independently. A parameter unit.directories indicates the version and the location 
of software associated with such units. Parameters bus_info_block and root.directory are stored at fixed locations, 
However, locations of the other blocks are specified by offset addresses. 

50 [0158] Fig. 16 shows details of bus.info.block, root directory, and unit_directries. 

[0159] In bus.info.block, CompanyJD is a parameter indicating an ID number of a manufacturer of a device. Chip. ID 
is a parameter indicating an ID which is uniquely assigned to the device and which is not used by any other device in 
the world. In accordance with the IEC1B33 standard, unit.specjd in unit.directory of a device, which satisfies the 
IEC1833 standard, is rewritten such that OOh is rewritten in a first octet, AOh in a second octet, and 2Dh in a third octet. 

55 On the other hand, unit.sw.version is rewritten such that 01 h is rewritten in a first octet and 1 is rewritten at a LSB 
(Least Significant Bit) of a third octet. 

[0160] Each node has a PCR (Plug Control Register) placed, in accordance with the IEC1883 standard, at addresses 
of 900h to 9FFh in the initial register space shown in Fig. 13. The PCR is a register for logically forming a signal path 
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analogous to an analog interface. That is, a concept of plug is realized by the PCR. 
[01 61 J Fig. 1 7 shows the structure of the PCR. 

[0162] The PCR includes an oPCR (output Plug Control Register) for indicating an output plug and an iPCR (input 
Plug Control Register) for indicating an input plug. The PCR also includes an oMPR (output Master Plug Register) for 

5 representing information associated with the output plug of the specific device and an iMPR (input Master Plug Register) 
for representing information associated with the input plug. Any IEEE1394 device can have only a single oMPR and 
a single iMPR but cannot have plural oMPRs or plural iMPRs. However, an IEEE1394 device may have plural oPCRs 
and iPCRs depending on the capacity of the IEEE1394 device. In the example shown in Fig. 17, the PCR includes 31 
oPCRs #0 to #30 and 31 iPCRs #0 to #30. The flow of isochronous data is controlled by controlling a register corre- 

10 sponding to a plug. 

[0163] Fig. 18 shows the structures of an oMPR, an oPCR, an iMPR, and an iPCR. 

[0164] Wherein the structure of the oMPR is shown in Fig. 18A, the structure of the oPCR in Fig. 18B, the structure 
of the iMPR in Fig. 18C, and the structure of the iPCR in Fig. 18D. 

[01 65] In a 2-bit field of "data rate capability" located on the MSB-side of the oMPR and in that of the iMPR, a code 
15 indicating the maximum isochronous data rate, at which the device is allowed to transmit, or receive data, is described, 
in a "broadcast channel base" field of the oMPR, a channel number used to output broadcast data is described. 
[01 66] In a 5-bit field of "number of output plugs" located on the LS B-side of the oMPR, a value indicating the number 
of output plugs, that is, oPCRs possessed by the device is described. In a 5-bit field of "number of input plugs" located 
on the LS B-side of the iMPR, a value indicating the number of input plugs, that is, iPCRs possessed by the device is 
20 described. A "non-persistent extension" field and a "persistent extension" field are reserved so that extension can be 
performed using these fields in the future. 

[0167] An "on-line" bit located at the MSB of the oPCR and an "on-line" bit located at the MSB of the iPCR indicate 
whether the plug is being used or not. If the "on-line" bit is equal to 1, the corresponding plug is in a ON-LINE state, 
while the plug is in an OFF-LI NE state when the "on-line" bit is equal to 0. A broadcast connection counter of the oPCR 
25 and that of the iPCR indicate whether there is a broadcast connection ( 1 ) or there is not broadcast connection (0). The 
value of a 6-bit point-to-point connection counter of the oPCR and that of the iPCR indicate the number of point-to- 
point connections associated with the corresponding plug. 

[0168] -The value of a 6-bit channel number of the oPCR and that of the iPCR indicate the isochronous channel 
number to which the corresponding plug is connected. The value of a 2-bit data rate of the oPCR indicates the actual 
30 data rate at which packets of isochronous data is output from the corresponding plug. The code of a 4-bit overhead ID 
of the oPCR indicates the overhead bandwidth of isochronous communication. The value of a 10-bit payload of the 
oPCR indicates the maximum value of data included in an isochronous packet, which can be handled by the corre- 
sponding plug. 

[0169] In the IEEE 1394 standard, an AV/C command set for controlling an IEEE 1394 device is defined. Thus, in the 
35 present embodiment; the master device 1 controls slave devices 2 using the AV/C command set. However, the master 
device 1 may also control slave devices 2 using a command set other than the AV/C command set. 
[0170] The AV/C command set is briefly described below. 

[0171] Fig. 19 shows the data structure of AV/C command set packet data transmitted in the asynchronous trans- 
mission mode. 

40 [0172] The AV/C command set is a command set for controlling an AV (Audio Visual) device. In a control system 
using the AV/C command set, an AV/C command frame and a response frame are transmitted between nodes in 
accordance with the FCP (Function Control Protocol). In order not to impose a large load on a bus and/or AV devices, 
a response to a command is returned in 100 ms. 

[0173] As shown in Fig. 19, asynchronous packet data includes 32 bits (1 quadlet) in the horizontal direction. A 
45 packet header of a packet is shown on the upper side of Fig. 19, and a data block is shown on the lower side. The 
destination of the data is indicated by destination. ID. 

[0174] CTS denotes the ID of a command set. In the case of the AV/C command set, CTS = "0000". When a packet 
is a command, the function type of the command is indicated by ctype/response. On the other hand, in the case in 
which a packet is a response, ctype/response indicates the result of a process performed in accordance with the 

50 command. Commands are generally classified into the following four types: (1 ) a command (CONTROL command) for 
controlling a function from the outside; (2) a command (STATUS command) for issuing a query about the status from 
the outside; (3) commands (GENERAL INQUIRY command and SPECIFIC INQUIRY command) for inquiring from the 
outside as to whether a CONTROL command is supported (wherein the GENERAL INQUIRY command is used to 
inquiry as to whether an opcode is supported, and the SPECIFIC INQUIRY command is used to inquiry as to whether 

55 an opcode and an operands are supported); and (4) commands (NOTIFY commands) for requesting transmission of 
a notification of a change in status to the outside. 

[0175] A response is returned depending on the type of a command. Responses which are returned in response to 
the control command include a NOT IMPLEMENTED response, an ACCEPTED response, a REJECTED response, 
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and an INTERIM response. Responses which are returned in response to the STATUS command include a NOT IM- 
PLEMENTED response, a REJECTED response, an IN TRANSITION response, and a STABLE response. Responses 
which are returned in response to the GENERAL INQUIRY command or the SPECIFIC INQUIRY command include 
an IMPLEMENTED response and a NOT IMPLEMENTED response. Responses which are returned in response to 
5 the NOTIFY command include a NOT IMPLEMENTED response, a REJECTED response, an INTERIM response, and 
a CHANGED response. 

[0176] A parameter "subunit type" is used to indicate the function of a device, such as a tape recorder/player or a 
tuner. When there two or more subunits of the same type, each subunit is identified by a subunit id (placed after "subunit 
type"), and addressing is performed using a subunit id. In a field of "opcode", a command is placed, and a parameter 
10 associated with the command is placed in a filed of "operand". In a field of "Additional operands", an additional operands 
are placed. In a field of "padding", dummy data is placed so that the packet length is adjusted to a predetermined 
number of bits. In a field of "data CRC (Cyclic Redundancy Check)", CRC for checking an error which can occur during 
transmission is placed. 

[0177] Fig. 20 shows specific examples of AV/C commands. 

15 [0178] Fig. 20A shows specific examples of ctype/response, wherein commands are shown on the upper side of the 
figure and responses are shown on the lower side. The CONTROL command is assigned "0000", and the STATUS 
command is assigned "0001". The SPECIFIC INQUIRY command is assigned "0010", and the NOTIFY command is 
assigned "0011". The GENERAL INQUIRY command is assigned "0100". "0101" to "0111" are reserved for future 
usage. The NOT IMPLEMENTED response is assigned "1000", and the ACCEPTED response is assigned "1001". 

'20 The REJECTED response is assigned "1010", and the IN TRANSITION response is assigned "1011". The IMPLE- 
MENTED/STABLE response is assigned "1100", and the CHANGED response is assigned "1101". The INTERIM re- 
sponse is assigned "1111". "1110" is reserved for future usage. 

[01 79] Fig. 20B shows specific examples of subunit types. "Video Monitor" is assigned "00000", and "Disk Recorder/ 
Player" is assigned "00011". "Tape Recorder/Player" is assigned "00100", and "Tuner" is assigned "00101". "Video 
25 Camera" is assigned "00111", and "Vender unique" is assigned "11100". "Subunit type extended to next byte" is as- 
signed "11110". "11111" is assigned "unit", which is used when a packet is transmitted to a device, for example, in order 
to turn on/off the power of the device. 

[0180] Fig. 20C shows specific examples of opcodes. There are opcode tables for respective subunit types, and 
opcodes for a device whose subunit type is Tape Recorder/Player are shown in Fig. 20C. An operand is defined for 

30 each opcode. In the examples shown in Fig. 20C, VENDOR-DEPENDENT is assigned "OOh", SEARCH MODE is 
assigned "50h". TIME CODE is assigned "51 h", and ATN is assigned "52h". OPEN MIC is assigned "60h", and READ 
MIC is assigned "61h". WRITE MIC is assigned "62h", and LOAD MEDIUM is assigned C1h" RECORD is 
assigned "C2h", PLAY is assigned "C3h", and WIND is assigned "C4h". 
[0181] Fig. 21 shows specific examples of an AV/C command and a response thereto. 

35 [0182] For example, to command a target device (consumer device, to be controlled) such as a playback device to 
perform a playback operation, a command shown in Fig. 21 A is transmitted from a controller (device which controls 
the target device) to the target device. In this case, the command is expressed using the AV/C command set, and thus 
CTS = "0000". Herein, ctype = "0000", because a CONTROL command is used to control the target device from the 
outside (Fig. 20A). Furthermore, subuntt type - "00100", because the device is a Tape Recoder/Player (Fig. 20B). In 

40 the specific example, ID is #0 and thus id = "000". Furthermore, the opcode is "C3h" specifying PLAY (Fig. 20C) and 
the operand is "75h" specifying FORWARD. If the playback operation is performed, the target device returns a response 
such as that shown in Fig. 21 B to the controller. In this specific example, the response is an "accepted" response 
indicating that the command is accepted, and thus response = "1001" (see Fig. 20A). The other parameters are the 
same as those shown in Fig. 21 A, and thus they are not described in further detail herein. 

45 [0183] In the scalable TV system, using the AV/C command set described above, various control operations are 
performed between the master device 1 and slave devices 2. Among control operations performed between the master 
device 1 and slave devices 2, those which are not supported by the AV/C command set are performed using commands 
and responses which are additionally defined. 

[0184] Further detailed information about IEEE1394 communication and the AV/C command set may be found, for 
50 example, in "WHITE SERIES No. 181 IEEE1394 MULTIMEDIA INTERFACE" published by Triceps. 

[0185] In the signal processor 137 of the master device 1 shown in Fig. 10 (and also in the signal processor 157 of 
a slave device 2 shown in Fig. 11), various kinds of digital signal processing are performed by the DSP 137A in ac- 
cordance with programs, as described earlier. One of such digital signal processing is to convert first video data into 
second video data. 

55 [01 86] In this video data conversion, in a case in which the first video data has a low resolution and the second video 
data has a high resolution, the video data conversion can be regarded as a process of increasing the resolution. In a 
case in which first video data with a low signal-to-noise ratio is converted into second video data with a high signal-to- 
noiae ratio, the video data conversion can be regarded as a noise reduction process. Furthermore, in a case in which 
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first video data with a particular image size is converted into second video data with a greater or smaller image size, 
the video data conversion can be regarded as a resizing process of resizing (enlarging or reducing) the image. 
[0187] Thus, various kinds of processing can be realized by the video data conversion, depending on the types of 
first and second video data. 

5 [0188] Fig. 22 shows an example of the functional structure of the signal processor 1 37 for performing the video data 
conversion. The functional structure shown in Fig. 22 may be implemented by means of a software program stored in 
the EEPROM 1 37B, that is, the functional structure may be realized by executing the software program by the DSP1 37A 
of the signal processor 137. 

[0189] In the signal processor 137 (Fig. 10), video data stored in the frame memory 127 or video data supplied from 
10 the CPU 129 is given as first video data to tap extractors 161 and 162. 

[0190] The tap extractor 161 employs pixels constituting second video data as a pixel of interest one by one and 
extracts, as prediction taps, some of pixels constituting first video data, to be used to predict the pixel value of the pixel 
of interest. 

[0191] More specifically, the tap extractor 1 61 extracts, as prediction taps, a plurality of pixels spatially or temporally 
is close to a pixel, corresponding to a pixel of interest, of the first video data (for example, the pixel, corresponding to the 
pixel of interest, of the first video data and pixels spatially or temporally adjacent to that pixel are extracted). 
[0192] The tap extractor 162 extracts, as class taps, some pixels from the first video data, to be used to classify the 
pixel of interest. 

[0193] Hereinafter, for simplicity, the prediction taps and the class taps are assumed to have the same tap structure, 
20 although the prediction taps and' the class taps may have different tap structures. 

[0194] The prediction taps extracted by the tap extractor 161 are supplied to a predictor 165, while the class taps 
extracted by the tap extractor 162 are supplied to a classifier 163. 

[0195] The classifier 163 classifies the pixel of interest on the basis of the class taps received from the tap extractor 
162 and supplies a class code indicating the determined class to a coefficient memory 164. 
25 [0196] The classification may be performed, for example, in accordance with an ADRC (Adaptive Dynamic Range 
Coding) algorithm. 

[0197] In the case in which the ADRC algorithm is used, pixel values of pixels extracted as class taps are subjected 
to the ADRC processing, and the class of the pixel of interest is determined in accordance with an ADRC code obtained 
via the ADRC processing. 

30 [0198] In a case in which K-bit ADRC is employed, the maximum value MAX and the minimum value MIN of the 
pixels values of pixels extracted as class taps are detected, and the local dynamic range of the set of pixels is determined 
as DR = MAC - MIN. The pixel values of the class taps are then requantized on the basis of the dynamic range DR. 
More specifically, the minimum value MIN is extracted from the pixel values of pixel of the class taps, and the resultant 
respective values are divided (quantized) by DR/2 K . The resultant K-bit pixel values of pixels of the class taps are 

35 arranged in a predetermined order into a bit string, and the resultant bit string is output as an ADRC code. For example, 
in a case in which 1-bit ADRC is employed, the minimum value MIN is subtracted from the pixel values of respective 
pixels of class taps, and the resultant values are divided by the mean value of the maximum value MAX and the 
minimum value MIN (the fractional portions are dropped), thereby converting the respective pixel values into 1-bit 
values (two-level values). The resultant 1-bit pixel values are then arranged in the predetermined order into a hit string, 

40 and the result is output as an ADRC code. 

[0199] Alternatively, the classifier 163 may directly output a level distribution pattern of pixel values of pixels of class 
taps as a class code. However, in this case, when class taps include pixel values of N pixels each represented in K 
bits, a class code output from the classifier 163 is selected from as many class codes as (2N) K , which is very huge. 
[0200] Thus, it is desirable that the classifier 163 perform classification after reducing the amount of information of 

45 class taps by means of the ADRC processing or vector quantization. 

[0201] Tap coefficients of respective classes, supplied from coefficient generator 166, are stored in the coefficient 
memory 164. Of those tap coefficients stored in the coefficient memory 164, a tap coefficient stored at an address 
corresponding to the class code supplied from the classifier 1 63 (a tap coefficient represented by the class code supplied 
from the classifier 163) is supplied to the predictor 165. 

50 [0202] The tap coefficients correspond to coefficients which are multiplied, in a digital filter, by input data at taps. 
[0203] The predictor 1 65 acquires the prediction taps output from the tap extractor 161 and the tap coefficients output 
from the coefficient memory 164 and determines a predicted value corresponding to the real value of the pixel of 
interest, using the prediction taps and the tap coefficients according to a predetermined prediction algorithm. Thus, 
the predictor 165 determines the pixel value (the predicted value) of the pixel of interest, that is, the pixel value of a 

55 pixel of the second video data, and outputs the result. 

[0204] The coefficient generator 1 66 generates tap coefficients for respective classes on the basis of the coefficient 
seed data stored in a coefficient seed data memory 167 and a parameter stored in a parameter memory 168. The 
resultant tap coefficients are overwritten in the coefficient memory 164. 
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[0205] The coefficient seed data memory 167 stores coefficient seed data for each class, wherein the coefficient 
seed data is obtained via learning of coefficient seed data as will be described later. The coefficient seed data refers 
to data serving as a seed in generating tap coefficients. 

[0206] When a parameter is supplied to the parameter memory 168 from the CPU 129 (Fig. 10) in response to an 
operation performed by a user on the remote commander 1 5, the parameter memory 1 68 stores the received parameter 
in the overwriting fashion. 

[0207] Now, referring to a flow chart shown in Fig. 23, video data conversion performed by the signal processor 137 
shown in Fig. 22 is described. 

[0208] The tap extractor 161 sequentially employs pixels constituting second video data, corresponding to first video 
data, as pixel of interest on a one-by-one basis. In step S1, parameter memory 168 determines whether a parameter 
has been supplied from the CPU 129. If it is determined that the parameter has been supplied, the process proceeds 
to step S2. In step S2, the parameter memory 168 stores the received parameter in an overwriting fashion. After 
completion of storing the parameter, the process proceeds to step S3. 

[0209] In the case in which it is determined in step S1 that the parameter has not been supplied from the CPU 129, 
the process jumps to step S3 without performing step S2. 

[0210] If the parameter from the CPU 129 is supplied to the parameter memory 168, that is, if a parameter input by 
a user by operating the remote commander 15, or if the parameter is set by the CPU 129, the content currently stored 
in the parameter memory 168 is replaced with the parameter supplied to the parameter memory 168. 
[0211] In step S3, the coefficient generator 166 reads coefficient seed data associated with each class from the 
coefficient seed data memory 167 and also reads a parameter from the parameter memory 168. The coefficient gen- 
erator 166 then determines tap coefficients for each class on the basis of the coefficient seed data and the parameter. 
The process then proceeds to step S4. In step S4, the coefficient generator 166 supplies the tap coefficients associated 
with each class to the coefficient memory 164, which stores the received tap coefficients in an overwriting fashion. The 
process then proceeds to step S5. 

[0212] In step S5, the tap extractors 161 and 162 extract prediction taps and class taps associated with the pixel of 
interest, respectively, from the first video data supplied to the tap extractors 161 and 162. The extracted prediction 
taps are supplied to the predictor 165 from the tap extractor 161, and the extracted class taps are supplied to the 
classifier 163 from the tap extractor 162. 

[0213] If the classifier 163 receives the class taps associated with the pixel of interest from the tap extractor 162, 
the classifier 163 classifies, in step S6, the pixel of interest on the basis of the class taps. Furthermore, the classifier 
163 outputs a class code indicating the determined class of the pixel of interest to the coefficient memory 164. There- 
after, the process proceeds to step S7. 

[0214] In step S7, the coefficient memory 164 reads a tap coefficient stored at an address corresponding to the class 
code supplied from the classifier 163 and outputs it. Furthermore, in step S7, the predictor 165 acquires the tap coef- 
ficient output from the coefficient memory 164. Thereafter, the process proceeds to step S8. 

[0215] In step 38, the predictor 165 performs a prediction operation according to a predetermined algorithm using 
the prediction taps output from the tap extractor 161 and the tap coefficient acquired from the coefficient memory 164. 
Thus, the predictor 165 then determines the pixel value of the pixel of interest and stores the resultant pixel value into 
the frame memory 127 (Fig. 10). Thereafter, the process proceeds to step S9. 

[021 6] In step S9, the tap extractor 1 6 1 determines whether the second video data includes one or more pixels which 
have not yet been taken as a pixel of interest. If it is determined in step S9 that the second video data includes such 
pixels, one of such pixels is taken as a next pixel of interest. The processing flow then returns to step S1 to repeat the 
process described above. 

[0217] On the other hand, if it is determined in step S9 that the second video data include no more such pixels which 
have not yet been taken as a pixel of interest, the process is ended. 

[0218] Steps S3 and S4 shown in Fig. 23 may be performed only when a new parameter value is overwritten in the 
parameter memory 168, and steps S3 and S4 may otherwise be skipped. 

[0219] Now referring to Fig. 22, the predicting operation performed by the predictor 165, the generation of tap coef- 
ficients by the coefficient generator 166, and the learning of coefficient seed data stored in the coefficient seed data 
memory 167 are described below. 

[0220] Herein we assume that video data having a high resolution (high-resolution video data) is employed as second 
video data, first video data having a low resolution (low-resolution video data) is produced by reducing the resolution 
of the high-resolution video data by means of filtering using a LPF (Low Pass Filter), and pixel values of high-resolution 
pixels are determined (predicted) by means of a prediction algorithm using prediction taps extracted from the low- 
resolution video data and using tap coefficients. 

[0221 ] In a case in which the prediction is performed in accordance with a linear prediction algorithm, the pixel value 
y of a high-resolution pixel may be determined, for example, by the following linear equation. 
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N 

y=Iw n X n 

n-1 (1) 

5 

where x n denotes the pixel value of an nth pixel, of prediction taps associated with the high-resolution pixel y, in low- 
resolution video data (hereinafter, such a pixel will be referred to as a low-resolution pixel), and w n denotes an nth tap 
coefficient multiplied by the nth low-resolution pixel (more strictly, the pixel value of the nth low-resolution pixel). In 
10 equation (1), it is assumed that the prediction tap includes N low-resolution pixels x,, x 2 ,..., x n . 

[0222] Alternatively, the pixel value y of a high-resolution pixel may be determined using a quadratic formula or a 
higher-order formula instead of the linear formula (1). 

[0223] In the example shown in Fig. 22, the coefficient generator 166 generates a tap coefficient w n from coefficient 
seed data stored in the coefficient seed data memory 167 and a parameter stored in the parameter memory 168. 
15 Herein, it is assumed that the coefficient generator 1 66 generates the tap coefficient w n in accordance with the following 
formula using the coefficient seed data and the parameter, 



m«1 (2 > 

. wherein p^ n denotes mth coefficient seed data used to determine the nth tap coefficient w n , and z denotes the 
25 parameter. According to equation (2), the tap coefficient w n is determined using M coefficient seed data p n 1( p n2 

[0224] Note that the equation used to determine the tap coefficients w n from the coefficient seed data p m n and the 
parameter z is not limited to equation (2). 

[0225] Herein, let us introduce a new variable ^ which is given by z"** 1 . That ie, variable t m is defined by the equation 
30 (3) using the parameter z appearing in equation (2). 

\ m =z^' (m=1,2,-,M) (3) 

35 [0226] Substituting equation (3) into equation (2) yields the following equation. 

M 

40 n=1 (4) 

[0227] According to equation (4), the tap coefficient w n can be determined by a linear formula of the coefficient seed 
data p n m and variable t^ 

[0228] Herein, let y k be the true value of a kth sample of a high-resolution pixel, and y k ' be a predicted value, of the 
45 true value y k , obtained using equation (1). The prediction error e^ is given by the following equation. 

e k=yk-yk* < 5 > 

50 [0229] In equation (5), the predicted value y k is determined in accordance with equation (1), and thus equation (5) 
can be rewritten as described below by replacing y k ' in accordance with equation (1). 
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where x n k denotes an nth low-resolution pixel of prediction taps associated with the klh sample of the high-resolution 
pixel. 

[0230] By substituting equation (4) into w n in equation (6), the following equation is obtained. 



(7) 



[0231] In an ideal case in which the prediction error e k given by equation (7) becomes 0, an optimum high-resolution 
pixel can be given using the coefficient seed data p n m . However, in general, it is difficult to determine such coefficient 
seed data p n m for all high-resolution pixels. 

[0232] The goodness of the coefficient seed data p n m can be evaluated, for example, by means of the least square 
15 method. That is, optimum coefficient seed data p n m can be obtained by minimizing the sum of squares of errors, E, 
given by the following equation. 



K _ 

k-1 (8 > 



where K denotes the number of samples (used in learning) of sets of a high-resolution pixel y k and low-resolution pixels 
25 x i,k» x 2,k» •••» x n,k constituting a prediction tap associated with the high-resolution pixel y k . 

[0233] The smallest (minimum) value of the sum of the squares of errors, E, given by equation (8) is obtained when 
the partial derivative of the sum E with respect to the coefficient seed data p n m becomes equal to 0 as shown in 
equation (9). 



(9) 



35 [0234] By substituting equation (6) into equation (9), the following equation is obtained. 

I t m Xru kek= £ t B Xn. k (yk" f £ ( £ Pm. ) X n , k) =0 
40 k=1 k=1 V \n-l\n=l J J no) 

[0235] Herein, X ( pJ q and Y ( p defined by equation (11) and (12). respectively, are introduced. 



K 

Xj.p. j.q= Z Xj.ktpXj.ktq 

k=] 



(i-1.2. ••••N:j=1,2. •••.N:p-1.2."-.ll:q-1.2.-".ID <ii) 



K 

Y,.p= Zxi.kVk 

K-1 (12) 
[0236] Thus, equation (10) can be rewritten into a normal equation using Xj p j q and Y, p as shown in (13). 
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[0237] The normal equation (13) can be resolved with respect to the coefficient see data p n m by means of, for 
example, the sweeping out method (Gauss-Jordan elimination method). 

20 [0238] In the signal processor 137 shown in Fig. 22, the coefficient seed data memory 167 stores coefficient seed 

data 0 n m obtained via learning of solving equation (13), in which a large number of high-resolution pixels y 1t y 2 y k 

are used as teacher data, and low-resolution pixels x 1 k , x 2 k x n k constituting a prediction tap associated with each 

high-resolution pixel y k . The coefficient generator 166 generates the tap coefficient w n from the coefficient seed data 
P n m and the parameter 2 stored in the parameter memory 168 in accordance with the equation (2). The predictor 165 

25 determines the pixel value (predicted value close to the true pixel value) of the pixel of interest of high-resolution pixels 
by calculating equation (1 ) using the generated tap coefficient w n and a low-resolution pixel (pixel of the first video 
data) X n of prediction taps associated with the pixel of interest 

[0239] Fig. 24 shows an example of the structure of a learning apparatus 1 37b for learning determining the coefficient 
seed data p n m by solving the normal equation given by (13). 
30 [0240] Video data used in learning of the coefficient seed data f3 n m is input to the learning apparatus 137b. As for 
the video data for learning, for example, high-resolution video data may be employed. 

[0241] In the learning apparatus 137b, the video data for learning is supplied to a teacher data generator 171 and a 
student data generator 173. 

[0242] The teacher data generator 171 generates teacher data from the received video data for learning and supplies 
35 the generated teacher data to a teacher data memory 172. That is, in this case, the teacher data generator 171 directly 
transfers, as the teacher data, the high-resolution video data given as the video data for learning to the teacher data 
memory 172. 

[0243] The teacher data memory 172 stores the high-resolution video data supplies as the teacher data from the 
teacher data generator 171. 

40 [0244] The student data generator 173 generates student data from the video data for learning and supplies the 
generated student data to a student data memory 1 74. More specifically, the student data generator 1 73 reduces the 
resolution of the high-resolution video data given as the video data for learning by means of filtering thereby generating 
low-resolution video data. The resultant low-resolution video data is supplied as the student data to the student data 
memory 174. 

45 [0245] To the student data generator 1 73, in addition to the video data for learning, some values, which the parameter 
to be supplied to the parameter memory 168 shown in Fig. 22 can take, are also supplied from the parameter generator 

180. For example, when the parameter z can take a real number in the range from 0 to z, z = 0, 1, 2 Z are supplied 

from the parameter generator 180 to the student data generator 173. 

[0246] The student data generator 1 73 generates low-resolution video data to be used as the student data by passing 
50 the high-resolution video data given as the video data for learning through a LPF (lowpass filter) with a cutoff frequent 
corresponding to each value of the parameter z. 

[0247] Thus, in this case, the student data generator 173 generates 2+1 low-resolution video data with different 
resolutions to be used as the student data from the high-resolution video data given as the video data for learning, as 
shown in Fig. 25. 

55 [0248] In the present example, the cutoff frequency of the LPF, through which the high-resolution video data is passed 
to generate the low-resolution video data used as the student data, increases with the value of the parameter z. There- 
fore, the resolution of the generated low-resolution video data increases with the value of the parameter z. 
[0249] In the present embodiment, for simplicity, it is assumed that the student data generator 173 generates low- 
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resolution video data by reducing the resolution of high-resolution video data by a factor corresponding to the parameter 
z in both horizontal and vertical directions. 

[0250J Referring again to Fig. 24, the student data memory 174 stores the student data supplied from the student 
data generator 1 73. 

[0251] The tap extractor 1 75 sequentially takes pixels of the high-resolution video data serving as the teacher data 
stored in the teacher data memory 172 and employs each pixel as a teacher pixel of interest on a one-by-one basis. 
For each teacher pixel of interest, the tap extractor 175 extracts low-resolution pixels of low-resolution video data from 
those stored as the student data in the student data memory 174 and produce prediction taps having the same tap 
structure as that produced by the tap extractor 161 shown in Fig. 22. The resultant prediction taps are supplied to an 
adder 178. 

[0252] For each teacher pixel of interest, the tap extractor 1 76 extracts low-resolution pixels of low-resolution video 
data from those stored as the student data in the student data memory 174 and produce class taps having the same 
tap structure as that produced by the tap extractor 162 shown in Fig. 22. The resultant class taps are supplied to a 
classifier 1 77. 

[0253] A parameter z generated by the parameter generator 180 is supplied to both tap extractors 175 and 176. 
Using generated student data corresponding to the parameter z supplied from the parameter generator 180 (more 
specifically, low-resolution video data produced, as the student data, using the LPF having a cutoff frequency corre- 
sponding to the parameter z), the tap extractors 175 and 1 76 produce prediction taps and class taps. 
[0254] The classifier 1 77 performs classification on the basis of the class taps output from the tap extractor 1 76, in 
a similar manner to the classifier 163 shown in Fig. 22. A class code indicating the determined class is output to the 
adder 178. 

[0255] The adder 178 reads a teacher pixel of interest from the teacher data memory 172 and performs addition 
processing on the teacher pixel of interest, the student data produced as the prediction taps associated with the teacher 
pixel of interest supplied from the tap extractor 1 75, and the parameter z applied in the production of the student data, 
for each class code supplied from the classifier 1 77. 

[0256] That is, the adder 178 acquires teacher data y k from the teacher data memory 172, a prediction tap Xj k (Xj k ) 
from the tap extractor 175, a class code from the classifier 177, and a parameter z, employed in production of student 
data used to produce the prediction tap, from the parameter generator 180. 

[0257] In order to determine components Xj pJ q of a matrix on the left side of equation (1 3), for each class indicated 
by the class code supplied from the classifier 177, the adder 178 determines the product, Xj k tpXj k t p , of the prediction 
tap (student data) Xj k (Xj k ) and the parameter z and then determines the sum of products thereby determining com- 
ponents Xj P j q according to equation (11). In this calculation, t p and tq in equation (11) are determined from the param- 
eter z according to equation (3). 

[0258] Furthermore, to determine components Y jp of a vector on the right side of equation (13), for each class indi- 
cated by the class code supplied from the classifier 1 77, the adder 1 78 determines the product, Xj k t p y k , of the prediction 
tap (student data) x i k and the teacher data y k and then determines the sum of products thereby determining components 
Y } p according to equation (12). In this calculation, tp in equation (12) is determined from the parameter z according to 
equation (3). 

[0259] The adder 178 stores, in its internal memory (not shown), the calculated components Xj pj q of the matrix on 
the left side of equation (13), determined for teacher data employed as the teacher pixel of interest and also stores the 
calculated components Y ip of the vector on the right side of equation (1 3). The adder 1 78 then calculates components 
Xi.kVi,*^ and x i,ktp v k' respectively, for teacher data newly employed as the teacher pixel of interest, using the teacher 
data y k , the student data Xj k (x j k ), and the parameter z (summing is performed to determine components Xj p j q ac- 
cording to equation (11) and summing is performed to determine components Yj p according to equation (12)), and the 
adder 178 adds the calculated components to the components Xj pJq of the matrix and the components Y i p of the 
vector, respectively, currently stored in the memory. 

[0260] The adder 1 78 performs the addition processing described above by employing all teacher data stored in the 
teacher data memory 172 as the teacher pixel of interest for ail values, 0, 1,..., Z, of the parameter z thereby creating 
the normal equation shown in (13) for each class, and the adder 178 supplies the resultant normal equation to a 
coefficient seed data calculator 1 79. 

[0261] The coefficient seed data calculator 179 solves the normal equation supplied from the adder 178 for each 
class thereby determining the coefficient seed data p^ n for each class. The determined coefficient seed data p m n is 
output. 

[0262] The parameter generator 180 generates parameter values, z, in the allowable range, to be supplied to the 
parameter memory 168 shown in Fig. 22. For example, z = 0, 1, 2...., Z are generated and supplied to the student data 
generator 173. The parameter generator 180 also supplies the generated parameter z to the tap extractors 175 and 
176 and the adder 178. 

[0263] Now, referring to a flow chart shown in Fig. 26, the process (learning process) performed by the learning 
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apparatus shown in Fig. 24 is described below. 

[0264] First in step S21, the teacher data generator 171 and the student data generator 173 generate teacher data 
and student data, respectively, from the video data for learning and output the resultant teacher data and student data, 
In this case, the teacher data generator 171 directly outputs the video data for learning as the teacher data without 

5 performing any processing on it. The student data generator 171 receives the parameter z having Z + 1 values gen- 
erated by the parameter generator 180 and passes the video data for learning through LPFs having cutoff frequencies 
corresponding to Z + 1 values (0, 1,..., 2) of the parameter z supplied from the parameter generator 180 thereby 
generating Z + 1 student data associated with the teacher data (video data for learning) for each frame. 
[0265] The teacher data output from the teacher data generator 1 7 1 is supplied to the teacher data memory 1 72 and 

10 stored therein. The student data output from the student data generator 173 is supplied to the student data memory 
174 and stored therein. 

[0266] Thereafter, the process proceeds to step S22. In step S22, the parameter generator 180 sets the parameter 
z to an initial value, such as 0 and supplies it to the tap extractors 175 and 176 and the adder 178. The process then 
proceeds to step S23. In step S23, the tap extractor 1 75 reads teacher data, which has not yet been employed as the 

15 teacher pixel of interest, from the teacher data memory 1 72 and employs it as a new teacher pixel of interest. Further- 
more, in this step S23, the tap extractor 175 generates a prediction tap associated with teacher pixel of interest from 
student data which corresponds to the parameter z output from the parameter generator 180 and which is read from 
the student data memory 174 (that is, from student data generated by passing the video data for learning corresponding 
to the teacher data employed as the teacher pixel of interest through an LPF with a cutoff frequency corresponding to 

20 the parameter z), and the tap extractor 175 supplies the generated prediction tap to the adder 178. Still furthermore, 
in this step S23, the tap extractor 176 generates a class tap associated with the teacher pixel of interest from the 
student data which corresponds to the parameter z output from the parameter generator 180 and which is read from 
the studentidata memory 174, and the extractor 176 supplies the generated class tap to the classifier 177. 
[0267] In the next step S24, the classifiers 177 classifies the teacher pixel of interest on the basis of the class tap 

25 associated with the teacher pixel of interest and outputs a class code indicating the determined class to the adder 1 78. 
The process then proceeds to step S25. 

[0268] In step S25, the adder 1 78 reads a teacher pixel of interest from the teacher data memory 1 72 and calculates 
the components Xj k tpXj k tq of the matrix on the left side of equation (13) and the components Xj k t p y k of the vector on 
the right side, using the teacher pixel of interest, the prediction tap supplied from the tap extractor 175, and the pa- 

30 rameter z output from the parameter generator 180. the adder 178 then adds the components Xj k tpXj (k tq of the matrix 
and the components Xj k tpV k of the vector, calculated from the pixel of interest, the prediction tap, and the parameter 
z, to the components of the matrix and the components of the vector, corresponding to the class code supplied from 
the classifier 177, of those which have already been obtained. The process then proceeds to step S26. 
[0269] In step S26, the parameter generator 180 determines whether the parameter z output from the parameter 

35 generator 180 is equal to the maximum allowable value Z. If it is determined in step S26 that the parameter z output 
from the parameter generator 180 is not equal to the maximum value Z (that is, the parameter z is smaller than the 
maximum value Z), the process proceeds to step S27. In step S27, the parameter generator 180 increments the pa- 
rameter z by 1 and outputs the resultant parameter z having the new value to the tap extractors 175 and 176 and the 
adder 178. The processing flow then returns to step S23 to repeat the process descried above. 

40 [0270] In the case in which it is determined in step S26 that the parameter z is equal to the maximum value Z, the 
process proceeds to step S28. In step S28, the tap extractor 175 determines whether all teacher data stored in the 
teacher data memory 172 have been employed as the teacher pixel of interest. If it is determined in step S28 that the 
teacher data stored in the teacher data memory 172 include data which has not yet been employed as the teacher 
pixel of interest, the tap extractor 175 employs teacher data, which has not yet been employed as the teacher pixel of 

45 interest, as a new teacher pixel of interest. The process flow then the resolution of the high-resolution video data to a 
resolution corresponding to the parameter z is employed as student data, as shown in Fig: 25, and learning is performed 
to directly determine coefficient seed data p m n , which results in a minimum value for the sum of squares of errors of 
predicted value y given by the linear formula (1), from the student data x„ and the tap coefficient w n represented by 
the coefficient seed data p m n and the variable ^ corresponding to the parameter z according to equation (4). Alter- 

50 natively, the learning of the coefficient seed data {5 mn may be performed as described below with reference to Fig. 27. 
[0271] Also in the example ehown in Fig. 27, as in the example shown in Fig. 25, high-resolution video data is 
employed as teacher data for learning and low-resolution video data produced by reducing the horizontal and vertical 
resolutions of the high-resolution video data by passing the high-resolution video data through an LPF with a cutoff 
frequency corresponding to the parameter z is employed as student data. First, for each value of the parameter z (z 

55 = 0, 1 , .. . , Z), tap coefficients w n are determined, which result in zero for the sum of squares of errors of predicted values 
y of teacher data predicted using a linear formula (1) including tap coefficients w n and student data x n . Furthermore, 
in the example shown in Fig. 27, the determined tap coefficients w n are employed as teacher data and the parameter 
z is employed as student data, and learning is performed so as to determine coefficient seed data p m n which results 
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in a minimum value for the sum of squares of errors of predicted values y of the tap coefficients w n employed as the 
teacher data, predicted using the coefficient seed data 0 m n and the parameter z employed as the student data according 
to equation (4). 

[0272] More specifically, the tap coefficients w n which result in the smallest (minimum) value for the sum E, given by 
equation (8), of squares of errors of predicted values y of teacher data predicted using a linear prediction formula (1) 
can be obtained when the partial differential of the sum E with respect to the tap coefficient w n is equal to zero. That 
is, the following equation should be satisfied. 

BE <* e 1 <* e 2 <> e n 

a^ =e ia^ + e ^ + - + e * ^ = 0 < n=1 - 2 - "■•"> < 14 > 

[0273] If equation (6) is partially differentiated with respect to the tap coefficient w n> the following equation is obtained. 

de k de k Be^ 

= - x i."- 3^ =- x 2*. 5^ = ">W < k=1 - 2 - -• K > < 15 > 

[0274] From equations (14) and (15), the following equation is obtained. 



K K K 

ZekXi,k=0, ZekX2,k=0, - IekXN,k=0 

k-1 k*1 k=t (16) 

[0275] By substituting equation (6) into e k in equation (16), equation (16) can be rewritten into a normal equation (1 7). 



k»1 

, K 
k=1 



(lxM fk yk) 

k=1 

(17) 

[0276] As with the normal equation (13), the normal equation (1 7) can be solved for the tap coefficient w n by means 
of, for example, the sweeping out method (Gauss-Jordan elimination method). 

[0277] Thus, by solving the normal equation (17), the optimum tap coefficients w n (which result in the smallest value 

for the sum E of squares of errors) are determined for respective values of the parameter z (z = 0, 1 Z). 

[0278] In the present embodiment in which the tap coefficient is determined from the coefficient seed data ^ n and 
the parameter ^ corresponding to the parameter z in accordance with the equation (4), if the tap coefficient determined 
in such a manner is donated by w n \ the optimum tap coefficient w n is given by the coefficient seed data p n m which 
results in zero in the error e n between the optimum tap coefficient w n given by equation (18) and the tap coefficient w n * 
given by equation (4). However, in general, it is difficult to determine such coefficient seed data P n m for all tap coeffi- 
cients W n . 
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e n =W n -W n ' (18) 
[0279] Equation (18) can be rewritten as follows, using equation (4). 



(19) 

10 

[0280] If the goodness of the coefficient seed data p n m is expressed by the means of the least square method, the 
optimum coefficient seed data p nm can be determined by minimizing the sum E of squares of errors represented in 
the following equation. 

15 

H 

E=Ie n 2 

n=1 (20) 

20 [0281] The smallest (minimum) value of the sum E of squares of errors expressed in equation (20) is given by fj nm 
which results in zero in the partial differential of the sum E with respect to the coefficient seed data {J nm , as shown in 
equation (21). 



25 



3E ^ Jen 



(21) 



30 [0282] Substituting equation (19) into equation (21) yields the following equation. 



M*" (I. '**}-° 



35 m 



[0283] Herein, Xy and Yj defined by equation (23) and (24) are introduced. 

40 

Z 

X«.j= Ztitj (i=l, 2, -J) 

2=0 

45 

Yi=it,w n 
z«0 

50 

[0284] Equation (22) can be rewritten into a normal equation (25) using Xy and Y s . 
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[0285] The normal equation (25) can be resolved with respect to the coefficient see data p n m by means of, for 
example, the sweeping out method (Gauss-Jordan elimination method). 

[0286] Fig. 28 shows an example of the configuration of a learning apparatus for learning determining the coefficient 
seed data p n m by solving the normal equation (25). In Fig. 24, similar parts to those in Fig. 24 are denoted by similar 
reference numerals, and similar parts are not described herein in further detail. 

[0287] An adder 190 receives a class code associated with a teacher pixel of interest from a classifier 177 and a 
parameter z from a parameter generator 180. The adder 190 reads a teacher pixel of interest from a teacher data 
memory 172 and performs addition processing in terms of the teacher pixel of interest and the student data produced 
as the prediction taps associated with the teacher pixel of interest supplied from the tap extractor 175, for each class 
code supplied from the classifier 177 and for each value of the parameter z output from the parameter generator 180. 
[0288] That is, the adder 190 acquires teacher data y k from the teacher data memory 172, a prediction tap x^ from 
the tap extractor 1 75, a class code from the classifier 1 77, and a parameter z, employed in production of student data 
used to produce the prediction tap Xp k , from the parameter generator 180. 

[0289] The adder 190 determines the product, x n k x n . k , of the student data and then determines the sum of the 
products thereby determining components of the matrix on the left side of equation (17), for each class indicated by 
the class code supplied from the classifier 1 77 and for each value of the parameter z output from the parameter gen- 
erator 180, using the prediction tap (student data) x„ k . 

[0290] Furthermore, for each class indicated by the class code supplied from the classifier 177 and for each value 
of the parameter z output from the parameter generator 1 80, using the prediction tap (student data) x,, k and the teacher 
data y k , the adder 190 determines the product, x n k y k , of the student data x n k and the teacher data y k and further 
determines the sum of the products thereby determining components of the vector on the right side of equation (17). 
[0291] The adder 190 stores, in its internal memory (not shown), the calculated components (Lx n k x n . k ) of the matrix 
on the left side of equation (17), determined for teacher data employed previously as the teacher pixel of interest and 
also stores the calculated components (Zx n k y k ) of the vector on the right side of equation (17). The adder 190 then 
adds (determines the sum in equation (17)) those components (Ix n k x n . k ) of the matrix and the components (£x n k y k ) 
of the vector with the corresponding (£x n k+1 x n . k+1 ) of the matrix and the components (Zx n k y k+1 ) of the vector, respec- 
tively, calculated, using the teacher data y k+1 and the student data X^.,, for teacher data newly employed as the 
teacher pixel of interest. 

[0292] The adder 1 90 performs the addition processing described above by employing all teacher data stored in the 
teacher data memory 1 72 as the teacher pixel of interest thereby creating the normal equation shown in (1 7) for each 
class and for each value of the parameter z, and the adder 190 supplies the resultant normal equation to a coefficient 
seed data calculator 191. 

[0293] The coefficient seed data calculator 191 solves the normal equation supplied from the adder 190 for each 
class and for value of the parameter z thereby determining the optimum tap coefficients w n for each value of the 
parameter z and for each class. The resultant optimum tap coefficients w n are supplied to the adder 1 92. 
[0294] The adder 192 performs the addition in terms of the parameter z (variable t m corresponding to the parameter 
z) and the optimum tap coefficients w n , for each class. 

[0295] That is, using the variable tj (tj) determined from the parameter z according to equation (3), the adder 192 
determines the products (tjtj) of variables tj and tj corresponding to the parameter z to be used to determine the com- 
ponents Xy, defined by equation (23), of the matrix on the left side of equation (25), and the adder 1 92 further determines 
the sum of the products, for each class. 

[0296] The components depend on only the parameter z and do not depend on the class. Therefore, in practice, 
it is not necessary to calculate the components Xy for each class. That is, the calculation of the components X^ is 
performed only once. 

[0297] Furthermore, using the variable tj determined from the parameter z according to equation (3) and also using 
the optimum tap coefficients w n , the adder 192 determines the products (tjW n ) of the variable tj corresponding to the 
parameter z and the optimum tap coefficient w n , to be used to determine the components Yj, defined by equation (24), 
of the vector on the right side of equation (25), and the adder 192 further determines the sum of the products, for each 
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class. 

[0298] If the components Xy defined by equation (23) and the components Yj defined by equation (24) have been 
determined thereby creating the normal equation (25), the adder 192 supplies the resultant normal equation to the 
coefficient seed data calculator 193. 
5 [0299] The coefficient seed data calculator 1 93 solves the normal equation (25) supplied from the adder 1 92 for each 
class thereby determining the coefficient seed data p^ n for each class. The determined coefficient seed data p m n is 
output. 

[0300] The coefficient seed data p^n determined in the above-described manner may be stored in the coefficient 
seed data memory 167 of the signal processor 137 shown in Fig. 22. 

10 [0301] Alternatively, the signal processor 137 shown in Fig. 22 may not include the coefficient seed data memory 
167 but the signal processor 137 may store the optimum tap coefficients w n output from the tap coefficient calculator 
191 shown in Fig. 28 for each value of the parameter, select an optimum tap coefficient depending on the parameter 
z stored in the parameter memory 168, and set the selected optimum tap coefficient into the coefficient memory 164. 
However, in this case, the signal processor 137 has to have a memory having a large capacity proportional to the 

15 number of values the parameter z can take. In contrast, in the case in which the coefficient seed data memory 167 is 
provided to store the coefficient seed data, the required storage capacity of the coefficient seed data memory 167 does 
not depends on the number of values the parameter z can take, and thus a memory having a small storage capacity 
can be employed as the coefficient seed data memory 167. In the case in which the coefficient seed data p m n is stored, 
the tap coefficient w n can be generated from the coefficient seed data p m n and the value of the parameter z in accord- 

20 ance with equation (2). This makes it possible to obtain a continuous value for the tap coefficient w n depending on the 
value of the parameter z, and thus it becomes possible to continuously adjust the image quality of the high-resolution 
video data, output as the second video data from the predictor 165 shown in Fig. 22. 

[0302] i In the case described above, because learning of the coefficient seed data is performed by employing the 
video data for learning as the teacher data corresponding to second video data and also employing the low-resolution 
25 video data obtained by reducing the resolution of the video data for learning as the student data corresponding to first 
video data, the coefficient seed data can be used in the video data conversion from the first video data having low 
resolution to the second video data having improved resolution. That is, the coefficient seed data can be used in video 
data conversion for improving the resolution. 

[0303] Therefore, if the coefficient seed data is stored in the EEPROM 1 37A in the signal processor 1 37 of the master 

30 device 1 , and if the functions shown in Fig. 22 are realized, and furthermore if the program for the video data conversion 
according to the flow chart ehown in Fig. 23 is stored, the Signal processor 137 has the capability of improving the 
horizontal resolution and the vertical resolution of given video data depending on the parameter z. 
[0304] By properly selecting the student data corresponding to the first video data and the video data employed as 
the teacher data corresponding to the second video data, it is possible to obtain coefficient seed data optimized for 

35 use in various kinds of video data conversion. 

[0305] For example, if learning is performed by employing high-resolution video data as the teacher data and also 
employing, as the student data, video data obtained by superimposing noise corresponding to the parameter z on the 
high-resolution video data employed as the teacher data, it is possible to obtain coefficient seed data optimized for 
use in the video data conversion for converting first video data including noise to second video data including no noise 

40 (k>w noise). That is, coefficient seed data optimized for use in reducing noise is obtained. 

[0306] For example, if learning is performed by employing certain video data as the teacher data and employing, as 
the student data, video data obtained by reducing the number of pixels of the video data employed as the teacher data 
at a reduction rate corresponding to the parameter z, or if learning is per formed by employing video data with an image 
size corresponding to the parameter z and employing, as the teacher data, data obtained by reducing the number of 

45 pixels of the video data employed as the student data, at a particular reduction rate, coefficient seed data optimized 
for use in video data conversion for converting first video data into second video data with an increased or reduced 
image size is obtained. That is, coefficient seed data optimized for use in resizing is obtained. 

[0307] That is, by storing coefficient seed data for use in removing noise or coefficient seed data for use in resizing 
in the EEPROM 1 37A of the signal processor 1 37 of the master device 1 , it becomes possible for the signal processor 
50 137 to have the capability of removing noise from video data or resizing (enlarging or reducing) video data, in accord- 
ance with the parameter z. 

[0308] Although in the example described above, the tap coefficient w n is defined as P 1n z° + P 2t n z1 + » — » Pm.n zm_1 
as shown in (2), and the tap coefficient w n for use in improving the horizontal resolution and the vertical resolution 
depending on the parameter z is determined in accordance equation (2), the tap coefficient w n may be determined 
55 separately for use in improving the horizontal resolution and for use in improving the vertical resolution depending on 
independent parameters zx and zy. 

[0309] For example, the tap coefficient w n is defined not according to equation (2) but as n z x °Zy° + P 2 n z x 1z y° + 
P^Z^ZyO + P^V + P5,n z x°V + K»**% 2 + fh,n*x% 3 + Ps.nW + P 9 ,n z x 2z y 1 + PlO^V- ™d the variable t m 
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is defined not according to equation (3) but as t 1 = zx°zy°, t 2 = zx 1 zy°, t 3 = zx 2 zy°, t 4 = zx 3 zy°, tg = zx°zy 1 , tg = zx°zy 2 , 
t 7 = zx°zy 3 , t 8 = zx 1 zy 1 , tg = zx 2 zy\ and t 10 = zx 1 zy 2 . Also in this case, the tap coefficient w n can be finally expressed 
by equation (4). Therefore, if, learning is performed in the learning apparatus (Fig. 24 or 28) by employing, as the 
student data, video data produced by reducing the horizontal resolution and the vertical resolution of teacher data 
depending on the parameter zx and zy, respectively, thereby determining coefficient seed data p^ n , the resultant 
coefficient seed data p^ n can be used to determine the tap coefficients w n for use in independently improving the 
horizontal resolution and the vertical resolution depending on the independent parameters zx and zy. 
[0310] Furthermore, for example, if a parameter zt corresponding to temporal resolution is introduced in addition to 
the parameters zx and zy corresponding to the horizontal resolution and the vertical resolution, respectively, tap coef- 
ficients w n for use in independently improving the horizontal resolution, the vertical resolution, and the temporal reso- 
lution, depending on the independent parameters zx, zy, and zt. 

[0311] Furthermore, for use in resizing, as in improving the resolution, not only the tap coefficients w n for use in 
enlarging (or reducing) video data by the same factor corresponding to the parameter z in both horizontal and vertical 
direction but also tap coefficients w n for use in enlarging video data in the horizontal direction and in the vertical direction, 
independently, depending on the parameter zx and zy, respectively, can be obtained. 

[0312] Furthermore, if learning is performed in the learning apparatus (Fig. 24 or 28) by employing, as the student 
data, video data produced by reducing the horizontal resolution and the vertical resolution of teacher data depending 
on a parameter zx and further adding noise to the teacher data depending on a parameter zy thereby determining 
coefficient seed data p^, the resultant coefficient seed data p^ n can be used to determine the tap coefficients w n for 
use in improving the horizontal resolution and the vertical resolution depending on the parameter zx and removing 
noise depending on the parameter zy. 

[0313] The above-described capability of performing video data conversion is possessed by not only the master 
device but also slave devices 2. 

[0314] Fig. 29 shows an example of a functional configuration of the signal processor 1 57 for performing video data 
conversion in a slave device 2 (Fig. 11 ). As with the signal processor 137 shown in Fig. 22, the functions shown in Fig. 
29 can be realized by executing a program stored in the EEPROM 1 57B using the DSP 1 57A in the signal processor 1 57. 

[031 5] In Fig. 29, the signal processor 157 of the slave device 2 is made up of parts including a tap extractor 201 

and a parameter memory 208 similar to the tap extractor 161,..., and the parameter memory 168 in the signal processor 
137 (Fig. 22) of the master device 1, and thus further description is not provided herein. 

[0316] In the present embodiment, the coefficient seed data stored in the signal processor 137 of the master device 
1 and the coefficient seed data stored in the signal processor 157 of the slave device 2 are different, at least partially, 
from each other, although the same coefficient seed data may be stored in both the signal processor 137 of the master 
device 1 and the signal processor 158 of the slave device 2. 

[0317] For example, coefficient seed data for use in resizing and coefficient seed data for use in improving the res- 
olution are stored in the signal processor 137 of the master device 1, while coefficient seed data for use in resizing 
and coefficient seed data for use in removing noise are stored in the signal processor 1 57 of the slave device 2. 
[031 8] Alternatively, coefficient seed data for use in resizing may be stored in the signal processor 1 37 of the master 
device 1, while coefficient seed data for use in removing noise may be stored in the signal processor 157 of a certain 
slave device 2y and coefficient seed data for use in improving the resolution may be stored in the signal processor 157 
of another slave device 2^. 

[031 9] It is also possible to store a plurality of coefficient seed data for use in various kinds of processing in both the 
signal processor 137 of the master device 1 and the signal processor 157 of the slave device 2. However, in this case, 
it is needed to store the coefficient seed data for use in various kinds of processing in both the EEPROM 137B and 
EEPROM 157B. This causes the EEPROM 137B and the EEPROM 157B to need to have a high storage capacity, 
which results in an increase in cost of the master device 1 or the slave device 2. 

[0320] In the scalable TV system according to the present embodiment, because the master device 1 and the slave 
device 2 are connected to each other so that IEEE1394 communication is possible between them, it is possible to 
transfer coefficient seed data from the master device 1 to the slave device 2 or from the slave device 2 to the master 
device 1 by means of IEEE1394 communication. For example, in a case in which a slave device 2 having coefficient 
seed data for use in removing noise is connected to a master device 1 , the master device 1 can perform noise reduction 
by using the coefficient seed data for use removing noise acquired from the slave device 2, even if the master device 
1 does not have the coefficient seed data for use in removing noise. 

[0321] Thus, the number of processes executable by the master device 1 (and also the slave device 2) with the 
number of slave devices connected in the scalable TV system. That is, the performance of the master device 1 (and 
also the capability of the slave device 2) becomes higher with the number of slave devices. 

[0322] This makes it possible to use a low-capacity memory as the EEPROM 137B or the EEPROM 1 57B, and thus 
it is possible to reduce the cost of the master device 1 or the slave device 2. Furthermore, this motivates users to 
purchase not only a master device 1 but also additional slave devices 2 to enhance the total performance of the scalable 
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TV system by increasing the number of slave devices 2. When a user purchases a new additional slave device, existing 
slave devices 2 possessed by the user is necessary to perform processes using coefficient seed data of the slave 
devices 2. This prevents the existing slave devices 2 from being thrown out by the user, and thus the present technique 
contributes effective use of resources. 
5 [0323] In the present embodiment, the signal processor 157 of a slave device 2 does not perform any independent 
process, but, when the signal processor 157 of the slave device 2 receives a command via the CPU 149 (Fig. 11) from 
the master device 1 by means of IEEE1394 communication, the signal process 157 performs a process in accordance 
with the received command. 

[0324] Therefore, although the slave device 2. has not only a capability (TV capability) of displaying an image on the 
10 CRT 3 1 in accordance with a television broadcast signal received by the antenna and output an associated audio signal 
from the speaker units 32L and 32R and also a capability (special capability) of performing a process achieved by the 
signal -processor 157, only the TV capability can be used when the slave device 2 is singly operated, but the special 
capability cannot be used. That is, in order to use the special capability of the slave device 2, it is required that the 
slave device 2 is connected with the master device 1 so as to form a scalable TV system. 
15 [0325] Now, referring to a flow chart shown in Fig. 30, a process performed by the master device 1 shown in Fig. 10 
is described below. 

[0326] First, in step S41 , the CPU 129 determines whether connection of some device to the terminal panel 21 or 
reception of some command from the IEEE1394 interface 133 or the IR receiver 135 has occurred as an event. If it is 
determined that no event has occurred, the process returns to step S41. 
20 [0327] In the case in which it is determined in step S41 that connection of a device to the terminal panel 21 has 
occurs as an event, the process proceeds to step S42. In step S42, the CPU 129 performs authentication as will be 
described later with reference to Fig. 31. The process then returns to step S41. 

[0328]' To determine whether some device has been connected to the terminal panel 21 , it is needed to detect con- 
nection of the device to the terminal panel 21. The detection can be performed, for example, as described below. 

25 [0329] '. If a device is connected (via a IEEE 1394 cable) to an IEEE 1394 terminal 21y provided on the terminal panel 
21 (Fig. 3), the terminal voltage of the IEEE 1394 21^ changes. If the IEEE1394 interface 133 detects the change in 
the terminal voltage, the IEEE1394 interface 133 informs the CPU 129 that the terminal voltage has changed. In re- 
sponse to receiving the notification of the change in the terminal voltage from the IEEE1394 interface 133, the CPU 
129 determines that a new device has been connected to the terminal panel 21 . The CPU 129 can also detect discon- 

30 nection of some device from the terminal panel 21 in a similar manner. 

[0330] On the other hand, in the case in which it is determined in step S41 that reception of some command from 
the IEEE1394 interface 133 or the IR receiver 135 has occurred as an event, the process proceeds to step S43. In 
step S43, the master device 1 performs processing corresponding to the received command. The process then returns 
to step S41. 

35 [0331] Referring to a flow chart shown in Fig. 31, authentication performed, in step S42 shown in Fig. 30, by the 
master device 1 is described below. 

[0332] In the authentication performed by the master device 1 , verification as to whether the device newly connected 
to the terminal panel 21 (hereinafter, referred to simply as a connected device) is an authorized IEEE1 394 device and 
verification as to whether the IEEE 1394 device is a television set capable of operating as a master device or a slave 
40 device (as to whether the IEEE1394 device is a scalable television set) are performed. 

[0333] That is, in the authentication performed by the master device 1, first, in step S51, the CPU 129 controls the 
IEEE1394 interface 133 so as to transmit an authentication request command to request mutual authentication to a 
device connected to the master device 1 . The process then proceeds to step S52. 

[0334] In step S52, the CPU 129 determines whether a response to the authentication request command has been 
45 returned from the connected device. If it is determined in step S52 that a response to the authentication request com- 
mand has not been returned from the connected device, the process proceeds to step S53. In step S53, the CPU 129 
determines whether a timeout has occurred, that is, whether a predetermined period of time has elapsed since the 
authentication request command was transmitted. 

[0335] If it is determined in step S53 that a timeout has occurred, that is, if no response is returned from the connected 
50 device within the predetermined period of time after the transmission of the authentication request command to the 
connected device, the process proceeds to step S54. In step S54, the CPU 129 determines that the authentication 
has failed because the connected device is not an authorized IEEE1394 device. In this case, the CPU 129 sets the 
operation mode to a single device mode in which no data is transmitted between the master device 1 and the connected 
device. The process exits the authentication routine. 
55 [0336] In this case, no further transmission of either IEEE 1394 communication data or other data is performed be- 
tween the master device 1 and the connected device which is not an authorized IEEE1394 device. 
[0337] On the other hand, in the case in which it is determined in step S53 that timeout has not occurred, the process 
flow returns to step S52 to repeat the process described above. 
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[0338] If it is determined in step S52 that a response to the authentication request command has been returned from 
the connected device, that is, if the response from the connected device has been received by the IEEE1394 interface 
133 and then transferred to the CPU 129, the process proceeds to step S55. In step S55, the CPU 129 generates a 
random (pseudorandom) number R1 in accordance with a predetermined algorithm and transmits it to the connected 

5 device via the IEEE1394 interface 133. 

[0339] Thereafter, the process proceeds to step S56. In step S56, the CPU 129 determines whether an encrypted 
random number E'(R1) produced by encrypting the random number R1 transmitted in step S55 in accordance with a 
predetermined encryption algorithm (for example, a secret-key encryption algorithm such as DES (Data Encryption 
Standard) or FEAL (Fast data Encipherment Algorithm), or RC5) has been received from the connected device. 

10 [0340] If it is determined in step S56 that the encrypted random number E'(R1) has not been received from the 
connected device, the process proceeds to step S57. In step S57, the CPU 129 determines whether a timeout has 
occurred, that is, whether a predetermined period of time has elapsed since the random number R1 was transmitted. 
[0341] If it is determined in step S57 that a timeout has occurred, that is, if the encrypted random number E'(R1) has 
not been received within a predetermined period of time after the transmission of the random number R1 to the con- 

*5 nected device, the process proceeds to step S54. In step S54, the CPU 129 determines that the connected device is 
not an authorized IEEE1394 device, and the CPU 129 sets the operation mode to the single-device mode. The process 
then exits the authentication routine. 

[0342] On the other hand, in the case in which it is determined in step S57 that timeout has not occurred, the process 
flow returns to step S56 to repeat the process described above. 

20 [0343] On the other hand, if it is determined in step S56 that the encrypted random number E'(R1) has been received 
fro the connected device, that is, if the encrypted random number E'(R1) transmitted from the connected device has 
been received by the IEEE1 394 interface 1 33 and then transferred to the CPU 129, the process proceeds to step S58. 
In step S58, the CPU 129 encrypts the random number R1 generated in step S55 according to a predetermined en- 
cryption algorithm thereby generating an encrypted random number E(R1). The process then proceeds to step S59. 

25 [0344] In step S59. the CPU 129 determines whether the encrypted random number E'(R1) received from the con- 
nected device is identical to the encrypted random number E(R1) generated, in step S58, by the master device. 
[0345] If it is determined in step S59 that the encrypted random numbers E'(R1 ) and E(R1) are not identical to each 
other, that is, if the encryption algorithm (and also the private key used in the encryption, if necessary) employed by 
the connected device is different from the encryption algorithm encrypted by the CPU 129, the process proceeds to 

30 step S54. In step S54, the CPU 129 determines that the connected device is not an authorized IEEE1394 device, and 
the CPU 129 sets the operation mode to the single-device mode. The process then exits the authentication routine. 
[0346] In the case in which it is determined in step S59 that the encrypted random numbers E'(R1) and E(R1) are 
identical to each other, that is, when the encryption algorithm employed by the connected device is identical to the 
encryption algorithm encrypted by the CPU 129, the process proceeds to step S60. In step S60, the CPU 129 deter- 

35 mines whether a random number R2 generated by the connected device to authenticate the master device 1 has been 
received from the connected device. 

[0347] If it is determined in step S60 that the random number R2 has not been received, the process proceeds to 
step S61. In step S61, the CPU 129 determines whether a timeout has occurred, that is, whether a predetermined 
period of time has elapsed since the encrypted random numbers E'(R1) and E(R1) were determined, in step S59, to 

40 be identical to each other. 

[0348] If it is determined in step S61 that a timeout has occurred, that is, if the random number R2 has not been 
received from the connected device within a predetermined period of time, the process proceeds to step S54. In step 
S54, the CPU 129 determines that the connected device is not an authorized IEEE1394 device, and the CPU 129 sets 
the operation mode to the single-device mode. The process then exits the authentication routine. 

45 [0349] On the other hand, in the case in which it is determined in step S61 that timeout has not occurred, the process 
flow returns to step S60 to repeat the process described above. 

[0350] On the other hand, if it is determined in step S60 that the random number R2 transmitted from the connected 
device has been received, that is, if the random number R2 transmitted the connected device has been received by 
the IEEE1394 interface 133 and then transferred to the CPU 129, the process proceeds to step S62. In step S62, the 
so CPU 129 encrypts the random number R2 according to a predetermined encryption algorithm thereby generating an 
encrypted random number E(R1 ), and the CPU 1 29 transmits it to the connected device via the I EEE 1 394 interface 1 33. 
[0351 ] At the time at which the random number R2 is received, in step S60, from the connected device, the connected 
device is authenticated as an authorized I EEE 1394 device. 

[0352] Thereafter, the process proceeds to step S63. In step S63, the CPU 129 controls the IEEB1394 interface 133 
55 so as to transmit a capability information request command to request capability information and the device ID of the 
connected device, together with the device ID and the capability information of the master device itself, to the connected 
device. 

[0353] The device ID refers to a unique ID identifying a television set such as a master device 1 or a slave device 2. 
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[0354] The capability information refers to information about the capability of a device. More specifically, the capability 
information includes information indicating the type of coefficient seed data (the type of video conversion for which the 
coefficient seed data can be used), information indicating external commands the device can accept (such as a power 
on/off command, a volume adjustment command, channel selection command, a brightness control command, and a 

5 sharpness control command), information indicating whether the device has an OSD (On-Screen Display) capability, 
information indicating whether muting is possible, and information indicating whether sleeping is possible. The capa- 
bility information also includes Information whether the device can operate as a master device or a slave device. 
[0355] In the master device 1 , the device ID and the capability information may be stored in the EEPROM 130 or in 
vendor_dependent_information of the configuration ROM shown in Fig. 15. 

10 [0356] Thereafter, the process proceeds to step S64. In step S64, The CPU 129 receives, via the I E EE 1 394 interface 
133, the device ID and the capability information transmitted by the connected device in response to the capability 
information request command transmitted in step S63 to the connected device. The received device ID and capability 
information are stored into the EEPROM 130. The process then proceeds to step S65. 

[0357] In step S65, the CPU 129 determines whether the connected device is a slave device, on the basis of the 
15 capability information stored in the EEPROM 130. If it is determined in step S65 that the connected device is a slave 
device, that is, if the connected device is authenticated as a slave device, the process jumps to step S68 without 
performing steps S66 and S67. In step S68, the CPU 129 sets the operation mode into the special-capability-available 
mode in which the special capability is enabled, that is, a control command is transmitted to a slave device to make 
the slave device perform a process by means of the special capability. The process flow then returns from the current 
20 routine. 

[0358] On the other hand, if it is determined in step S65 that the connected device is not a slave device, the process 
proceeds to step S66. In step S66, the CPU 129 determines whether the connected device is a master device, on the 
basis of the capability information stored in the EEPROM 130. If it is determined in step S66 that the connected device 
is a master device, that is, if the connected device is authenticated as a master device, the process proceeds to step 
25 S67. In step S67, the CPU 129 performs master-slave arbitration with the connected device having the capability of 
serving as a master device. 

[0359]. That is, in this case, a device capable of serving as a master device is connected to the master device 1, and 
thus, of the television sets included in the scalable TV system, two television sets have the capability of serving as a 
master device. However, in the scalable TV system according to the present embodiment of the invention, it is required 
30 that only one television set should operate as a master device. In step S67, to meet the above requirement, the master- 
slave arbitration is performed to determine whether the master device 1 or the connected device having the capability 
of serving as a master device is to operate as a master device. 

[0360] For example, a master device which was incorporated into the scalable TV system at an earlier point of time; 
that is, the master device 1 in this specific example, is selected as the master device of the scalable TV system. The 

35 other device having the capability of serving as a master device is set to operate as a slave device. 

[0361] After completion of the master-slave arbitration in step S67, the process proceeds to step S68, in which the 
CPU 129 sets the operation mode to the special-capability-available mode. The process then exits the present routine. 
[0362] On the other hand, if it is determined in step S66 that the connected device is not a master device, that is, if 
the connected device is neither a master device nor a slave device, and thus if the connected device is authenticated 

40 as neither a master device nor a slave device, the process proceeds to step S69. In step S69, the CPU 129 sets the 
operation mode into an ordinary-capability-only mode in which control commands for performing operations of special 
capability are not allowed although ordinary AV/C commands can be transmitted between the master device and slave 
devices. Thereafter, the process returns from the present routine. 

[0363] In this case, because the connected device is neither a master device nor a slave device, the special capability 
45 is not provided to the device connected to the master device 1 . However, in this case, because the connected device 
is an authorized IEEE1394 device, transmission of ordinary AV/C commands between the master device 1 and the 
connected device is allowed. That is, in this case, any one of the master device 1 and the connected device can be 
controlled by the other device (or by another IEEE1394 device connected to the master device 1) using an ordinary 
AV/C command. 

50 [0364] The operation of the slave device 2 shown in Fig. 11 is described below with reference to a flow chart shown 
in Fig. 32. 

[0365] First, in step S71, the CPU 149 determines whether connection of some device to the terminal panel 41 or 
reception of some command from the IEEE1394 interface 153 or the IR receiver 155 has occurred as an event. If it is 
determined that no event has occurred, the process returns to step S71 . 
55 [0366] In the case in which it is determined in step S71 that connection of a device to the terminal panel 41 has 
occurred as an event, the process proceeds to step S72. In step S72, the CPU 149 performs authentication as will be 
described later with reference to Fig. 33. The process then returns to step S71. 

[0367] To determine whether some device has been connected to the terminal panel 41 , it is needed to detect con- 
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nection of the device to the terminal panel 41. The detection may be performed, for example, in a similar manner to 
step S41 in Fig. 30 described earlier. 

[0368] On the other hand, if it is determined in stepS71 that reception of some command from the IE EE 1394 interface 
153 or the IR receiver 155 has occurred as an event, the process proceeds to step S73. In step S73, the slave device 
2 performs processing corresponding to the received command. The process then returns to step S71. 
[0369] Now, referring to a flow chart shown in Fig. 33, the authentication processing performed in step S72 in Fig. 
32 by the slave device 2 is described below. 

[0370] In the authentication performed by the slave device 2, verification as to whether the device newly connected 
to the terminal panel 41 (hereinafter, referred to simply as a connected device) is an authorized IEEE1394 device and 
verification as to whether that IEEE 1394 device is a master device are performed. 

[0371] That is, in the authentication performed by the slave device 2, first, in step S81, the CPU 149 determines 
whether an authentication request command for performing authentication has been received from the connected 
device. If it is determined that the authentication request command has not been received, the process proceeds to 
step S82. 

[0372] In step S82, the CPU 1 29 determines whether a timeout has occurred, that is, whether a predetermined period 
of time has elapsed since the authentication process was started. 

[0373] If it is determined in step S82 that a timeout has occurred, that is, if the authentication request command has 
not been received from the connected device within a predetermined period of time after starting the authentication 
process, the process proceeds to step S83. In step S83, the CPU 149 determines that the authentication has failed 
because the connected device is not an authorized IEEE1394 device. In this case, the CPU 149 sets the operation 
mode to a single device mode in which data transmission with the connected device is not performed. The process 
then returns from the present routine. 

[0374] In this case, not only IEEE 1394 communication but also any other data transmission with the connected 
device, which is not an authorized IEEE1394 device, is not performed thereafter 

[0375] On the other hand, in the case in which it is determined in step S82 that timeout has not occurred, the process 
flow returns to step S81 to repeat the process described above. 

[0376] On the other hand, if it is determined in step S81 that the authentication request command transmitted from 
the connected device has been received, that is, if the authentication request command transmitted in step S51 in Fig. 
31 by the master device 1 serving as the connected device has been received by the IEEE1 394 interface 1 53 and then 
transferred to the CPU 149, the process proceeds to step S84. In step S84, the CPU 149 controls the IEEE1394 
interface 153 so as to transmit a response to the authentication request command to the connected device. 
[0377] Although in the present embodiment, steps S51 to S53 in Fig. 31 are performed by the master device 1 and 
steps S81 , S82, and S84 in Fig. 33 are performed by the slave device 2, steps S51 to S53 in Fig. 31 may be performed 
by the slave device 2 and steps S81 , S82, and S84 in Fig. 33 may be performed by the master device 1 . 
[0378] The process then proceeds to step S85. In step S85, the CPU 149 determines whether a random number R1 
has been received from the connected device. If it is determined that the random number R1 has not been recoived, 
the process proceeds to step S86. 

[0379] In step S86, the CPU 1 49 determines whether a timeout has occurred, that is, whether a predetermined period 
of time has elapsed since the response to the authentication request command was transmitted in step S84. 
[0380] If it is determined in step S86 that a timeout has occurred, that is, if the random number R1 has not been 
received within a predetermined period of time after the transmission of the response to the authentication request 
command, the process proceeds to step S83. In step S83, the CPU 149 determines that the connected device is not 
an authorized IEEE1394 device, In this case, the CPU 129 sets the operation mode to a single device mode in which 
no data is transmitted between the master device 1 and the connected device. The process then returns from the 
present routine. 

[0381] On the other hand, in the case in which it is determined in step S86 that timeout has not occurred, the process 
flow returns to step S85 to repeat the process described above. 

[0382] On the other hand, if it is determined in step S85 that the random number R1 transmitted from the connected 
device has been received, that is, if the random number R1 transmitted in step S55 in Fig. 31 by the master device 1 
serving as the connected device has been received by the IEEE1394 interface 153 and then transferred to the CPU 
149, the process proceeds to step S87. In step S87, the CPU 149 encrypts the random number R1 according to a 
predetermined encryption algorithm thereby generating an encrypted random number E'(R1). Furthermore, in this step 
S87, the CPU 149 controls the IEEE 1394 interface 153 so as to transmit the encrypted random number E'(R1) to the 
connected device. Thereafter, the process proceeds to step S89. 

[0383] In step S89, the CPU 149 generates a random (pseudorandom) number R2 and controls the IEEE1394 in- 
terface 153 so as to transmit the generated random number R2 to the connected device. The process then proceeds 
to step S90. 

[0384] In step S90, the CPU 149 determines whether an encrypted random number E(R2) produced, in step S62 in 
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Fig. 31 , by the master device 1 serving as the connected device by encrypting a random number R2 has been received 
from the connected device. 

[0385] If it is determined in step S90 that the encrypted random number E(R2) transmitted from the connected device 
has not been received, the process proceeds to step S91. In step S91, the CPU 149 determines whether a timeout 

5 has occurred, that is, whether a predetermined period of time has elapsed since the random number R2 was transmitted, 
[0386] If it is determined in step S91 that a timeout has occurred, that is, if the encrypted random number E(R2) has 
not been received within a predetermined period of time after the transmission of the random number R2 to the con- 
nected device, the process proceeds to step S83. In step S83, the CPU 149 determines that the connected device is 
not an authorized IE EE 1394 device, and the CPU 149 sets the operation mode to the single-device mode. The process 

10 then returns from the present routine. 

[0387] On the other hand, in the case in which it is determined in step S91 that timeout has not occurred, the process 
flow returns to step S90 to repeat the process described above. 

[0388] On the other hand, if it is determined in step S90 that the encrypted random number E(R2) transmitted from 
the connected device has been received, that is, if the encrypted random number E(R2) transmitted from the connected 
15 device has been received by the IEEE1 394 interface 153 and then transferred to the CPU 149, the process proceeds 
to step S92. In step S92, the CPU 149 encrypts the random number R2 generated in step S89 according to a prede- 
termined encryption algorithm thereby generating an encrypted random number E'(R2). The process then proceeds 
to step S93. 

[0389] In step S93, the CPU 149 determines whether the encrypted random number E(R2) received from the con- 
20' nected device is identical to the encrypted random number E'(R2) generated, in step S92, by the slave device. 

[0390] If it is determined in step S93 that the encrypted random numbers E(R2) and E'(R2) are not identical to each 
other, that is, if the encryption algorithm (and also the private key used in the encryption, if necessary) employed by 
the connected device is different from the encryption algorithm encrypted by the CPU 149, the process proceeds to 
step S83. In step S83, the CPU 149 determines that the connected device is not an authorized IE EE 1394 device, and 
25 the CPU 149 sets the operation mode to the single-device mode. The process then exits the authentication routine. 
[0391] , On the other hand, if it is determined in step S93 that the encrypted random numbers E(R2) and E'(R2) are 
identical to each other, that is, when the encryption algorithm employed by the connected device is identical to the 
encryption algorithm encrypted by the CPU 1 49 and thus the connected device has been authenticated as an authorized 
IEEE 1394 device, the process proceeds to step S94. In step S94, the CPU 149 receives, via the IEEE 1394 interface 
30 153, a device ID and capability information transmitted together with a capability information request command trans- 
mitted in step S63 in Fig. 31 by the master device 1 serving as the connected device. The received device ID and 
capability information are stored in the EEPROM 150. 

[0392] The process then proceeds to step S95. In step S95, in response to the capability information request com- 
mand received in step S94 from the connected device, the CPU 149 controls the IEEE 1394 interface 153 so as to 
35 transmit the device ID and the capability information of the slave device 2 to the connected device, The process then 
proceeds to step S96. 

[0393] In the slave device 2, as with the master device 1 described earlier with reference to Fig. 31, the device ID 
and the capability information may be stored in the EEPROM 1 50 or vendor_dependent_information of the configuration 
ROM shown in Fig. 15. 

40 [0394] In step S96, the CPU 149 determines whether the connected device is a master device, on the basis of the 
capability information stored in the EEPROM 1 50. If it is determined in step S96 that the connected device is a master 
device, that is, if the connected device is authenticated as a master device, the process proceeds to step S97. In step 
S97, the CPU 149 sets the operation mode into the special-capability-available mode in which the special capability 
is enabled, that is, a control command transmitted from the master device serving as the connected device is acceptable 

45 and a process specified by the command can be performed. The process then returns from the present routine. 

[0395] If the operation mode of the slave device 2 is set to the special-capability-available mode, commands input 
via the front panel 154 or the IR receiver 155 of the slave device 2 are basically ignored, and only commands received 
from the master device 1 via the IEEE1394 interface 153 are accepted. For example, the channel selection or the 
sound volume control in the slave device 2 is performed in accordance with a command issued by the master device 

50 1. in this sense, the scalable TV system can be regarded as a centralized control system in which all slave devices 2 
in the scalable TV system are controlled by the master device 1 . 

[0396] Transmission of a command from the master device 1 (Fig. 10) to a slave device 2 may be performed in 
response to inputting via the front panel 134 or the IR receiver 135, or may be performed in such a manner that a 
command input via the front panel 154 or the IR receiver 155 of the slave device 2 is transferred to the master device 
55 1 via the IEEE1394 interface 153 and further transferred from the master device 1 back to the slave device 2. 

[0397] On the other hand, if it is determined in step S96 that the connected device is not a master device, that is, if 
authentication of the connected device as a master device fails, the process proceeds to step S98. In step S98, the 
CPU 149 sets the operation mode into an ordinary-capability-only mode in which control commands for performing 
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operations of special capability are not allowed although ordinary AV/C commands can be transmitted between the 
master device and slave devices. The process then returns from the present routine. 

[0398] In this case, because the device connected to the slave device 2 is not a master device, the connection does 
not cause the special capability to be provided. That is, when a slave device is connected to the slave device 2, the 
special capability is not provided. However, in this case, because the connected device is an authorized IEEE1394 
device, transmission of ordinary AV/C commands between the slave device 2 and the connected device is allowed. 
That is, in this case, any one of the slave device 2 and the connected device (and also other slave devices) can be 
controlled by the other device using an ordinary AV/C command. 

[0399] If the authentication described above with reference to Figs. 31 and 33 are successful in the master device 
1 and the slave device 2, and if the operation mode of the master device 1 and that of the slave device 2 are set to the 
special-capability-available mode, processes by means of the special capability of the scalable TV system are per- 
formed in step S43 in Fig. 30 and in S73 in Fig. 32, in the master device 1 and the slave device 2, respectively, as 
described below. 

[0400] In the master device 1 , as described earlier with reference to Fig. 10, an image and a sound/voice of a television 
broadcast program are output (that is, the image is displayed and the sound/voice is output). When such an image 
and a sound/voice are being output in the master device 1, if a user presses a guide button switch 63 of the remote 
commander 15 (Fig. 7) (or a guide button switch 93) of the remote commander 35 (Fig. 8)), a infrared ray is emitted 
from the remote commander 15 in response to the operation performed by the user. The infrared ray is received by 
the IR receiver 135 of the master device 1 (Fig. 10), and a command corresponding to the operation performed on the 
guide button switch 63 (hereinafter, referred to as a caption display command) is supplied to the CPU 129. 
[0401] Although the infrared ray emitted from the remote commander 15 is also received by the IR receiver 155 of 
the slave device 2 (in Fig. 11), the slave device 2 ignores the received infrared ray. 

[0402] If the CPU 129 of the master device 1 (Fig. 10) receives the caption display command, the CPU 129 of the 
master device 1 performs processing associated with the closed caption in accordance with the algorithm shown in 
the flow chart of Fig. 34. 

[0403] That is, first, in step S101 , the CPU 129 determines whether the transport stream being supplied to the de- 
multiplexer 124 includes closed caption data. 

[0404] When closed caption data is incorporated into an MPEG video stream, the closed caption data is placed, for 
example, as MPEG user data (MPEG-2 user data) in the sequence layer of the MPEG video stream. In this case, in 
step S101, the CPU 129 examines the transport stream being supplied to the demultiplexer 124 to determine whether 
closed caption data is included in the transport stream. 

[0405] If it is determined in step S101 that the transport stream does not include closed caption data, the closed 
caption processing is terminated without performing the following process. 

[0406] However, if it is determined in step S101 that the transport stream includes closed caption data, the process 
proceeds to step S102, and the CPU 129 checks the capability information, stored in the EEPROM 1 30, of the master 
device 1 and the slave devices included in the scalable TV system to detect a television set having coefficient seed 
data for closed caption from those of the scalable TV system. As described earlier, the capability information includes 
information indicating the type of coefficient seed data of each television set of the scalable TV system, and thus, in 
the present step S102, a television set having coefficient seed data for closed caption is retrieved by checking the 
capability information. 

[0407] The coefficient seed data for closed caption refers to coefficient seed data obtained by means of learning in 
which video data of a closed caption displayed in accordance with closed caption data is used as teacher data while 
video data obtained by reducing the resolution of the teacher data, video data obtained by adding noise to the teacher 
data, or video data obtained by reducing the image size of the teacher data is used as student data. The coefficient 
seed data obtained via such learning is suitable for improving the resolution, removing noise, or enlarging the image 
size of a closed caption image. 

[0408] The process proceeds to step S1 03. lnstepS103, the CPU 129 determines whether there is a television set 
having coefficient seed data for use in dealing with closed captions, on the basis of the result of the retrieval in step S1 02. 
[0409] If it is determined in step S1 03 that there is no television set having coefficient seed data dedicated to usage 
in dealing with closed captions, the process proceeds to step S104. In step 5104, the CPU 129 controls the signal 
processor 137 so as to start displaying closed captions in a normal mode. 

[0410] The signal processor 137 also has the capability of operating as an ordinary closed caption decoder. Thus, 
the CPU 1 29 requests the demultiplexer 124 to supply closed caption data included in the transport stream and transfers 
the closed caption data, supplied by the demultiplexer 124 in response to the request, to the signal processor 137. 
The signal processor 137 decodes the closed caption data received from the CPU 129 and superimposes the obtained 
closed caption on the video data stored in the frame memory 127, at a specified location of the video data. As a result, 
video data including the video data decoded by the MPEG video decoder 125 and the closed caption superimposed 
thereon is displayed on the CRT 11. 
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[041 1] Thus, in this case, a content image and a corresponding closed caption superimposed on the content image 
are displayed on the CRT 11 of the master device 1, as with a usual television set having a built-in closed caption 
decoder. 

[041 2] If the displaying of the closed caption is started, the process proceeds to step S1 05. In step S1 05, as in step 
5 S101, the CPU 129 determines whether the transport stream being supplied to the demultiplexer 124 includes more 
closed caption data to be displayed. 

[041 3] If it is determined in step S 1 05 that no closed caption data is included, the process jumps to step S1 07 without 
performing step S106. In step S107, the CPU 129 controls the signal processor 137 so as to stop the decoding of the 
closed caption data. Thus, the process exits the closed caption processing routine. 
10 [0414] On the other hand, if it is determined in step S105 that the transport stream being supplied to the demultiplexer 
124 includes more closed caption data to be displayed, the process proceeds to step S106. In step S106, the CPU 
129 determines whether a command to terminate displaying the closed caption (hereinafter, referred to as a closed 
caption display terminate command) has been received. 

[0415] If it is determined in step S106 that the closed caption display terminate command has not been received, 
15 the process flow returns to step S 105 to repeat the process described above. That is, in this case, displaying of the 
closed caption is continued. 

[0416] On the other hand, if it is determined in step S106 that the closed caption display terminate command has 
been received, that is, if the IR receiver 135 has received an infrared ray, corresponding to the closed caption display 
terminate command, emitted from the remote commander 15 in response to a turning-off operation, performed by a 
20 user, on the guide button switch 63 of the remote commander 15 (Fig. 7) (or the guide button switch 93 of the remote 
commander 35 (Fig. 8)), the process proceeds to step S107. In step S107, the CPU 129 controls the signal processor 
137, as described above, so as to stop the decoding of the closed caption data. Thus, the process exits the closed 
caption processing routine. 

[0417]r- On the other hand, if it is determined in step S103 that there is a television set serving as a slave device and 
25 having coefficient seed data for use in dealing with closed captions (hereinafter, such a device will be referred to as a 
slave device having caption coefficient seed data), the process proceeds to step S108. In step S108, the CPU 129 
selects a slave device to be used to display closed captions from television sets serving as slave devices in the scalable 
TV system. 

[0418] For example, the CPU 1 29 selects a slave device 2 2 3 located on the left side of the master device 1 or a slave 
30 device 2 32 located on the lower side of the master device 1 as the slave device used to display closed captions (here- 
inafter, such a slave device will be referred to as a slave device for displaying captions). As described above, the master 
device 1 has information about the locations of respective slave devices 2y relative to the location of the master device 
1 , and the master device 1 identifies the slave devices such as a slave device 223 located on the left side of the master 
device 1, a slave device 2 32 located under the master device 1, and so on, on the basis of the information about the 
35 locations of slave devices 2 t y 

[0419] The process then proceeds to step S109. In step S109, the CPU 129 transmits, via the IEEE1394 interface 
133, a command to the slave device having the caption coefficient seed data to request it to return the coefficient seed 
data dedicated to usage in dealing with closed captions. 

[0420] In the above process, the CPU 129 identifies the slave device having caption coefficient seed data by its 
40 device ID stored, together with capability information, in the EEPROM 130 and the CPU 129 specifies the destination 
of transmission of the command to request the coefficient seed data for use in dealing with dosed captions (hereinafter, 
such a command will be referred to as a coefficient seed data request command) by the device ID. Note that when the 
CPU 129 transmits other commands to a slave device, the CPU 129 identifies the slave device by its device ID and 
specifies the destination by the device ID. 
45 [0421] In step S109, the CPU 129 receives, via the IEEE1394 interface 133, the coefficient seed data for use in 
dealing with closed captions transmitted from the slave device having the caption coefficient seed data in response to 
the coefficient seed data request command, thereby acquiring the coefficient seed data for use in dealing with closed 
captions. 

[0422] In a case in which the coefficient seed data for use in dealing with closed captions is stored in the EEPROM 
50 137B of the signal processor 137 of the master device 1 itself, the acquisition of the coefficient seed data for use in 
dealing with closed captions in step S109 is performed by the CPU 129 by reading it from the EEPROM 137B. 
[0423] Even when coefficient seed data for use in dealing with closed captions is not stored in any television set of 
the scalable TV system, if coefficient seed data dedicated to usage in dealing with closed caption is available from a 
coefficient seed data server (not shown), the CPU 129 may acquire the coefficient seed data for use in dealing with 
55 closed captions by accessing the coefficient see data server via the modem 136. 

[0424] Not only such coefficient seed data dedicated to usage in dealing with closed caption but also coefficient seed 
data for use in other processes (video data conversion) descried later may also be acquired in a similar manner. 
[0425] Providing of coefficient seed data from the coefficient seed data provider may or may not be free of charge. 
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[0426] After acquiring the coefficient seed data for use in dealing with closed captions in step S109, the CPU 129 
controls, in next step S110, the IEEE1394 interface 133 so as to transmit dosed caption display command, together 
with the coefficient seed data for use in dealing with closed captions, to the slave device for displaying closed captions 
to command it to display the closed caption. The process then proceeds to step S111. 

[0427] In step S111, the CPU 129 transmits, via the IEEE1394 interface 133, an external input select command to 
the slave device for displaying captions to command it to select data input to the IEEE1394 interface 153 (Fig. 11) and 
display the input data on the CRT 31. The process then proceeds to step S112. 

[0428] In step S112, the CPU 129 starts transferring closed caption data to the slave device for displaying captions, 
[0429] That is, the CPU 129 requests the demultiplexer 124 to supply closed caption data included in the transport 
stream. In response to the request, closed caption data is supplied from the demultiplexer 124 to the CPU 129. The 
CPU 129 controls the IEEE1394 interface 133 so as to transfer closed caption data received to the demultiplexer 124 
to the slave device for displaying captions. 

[0430] If the transferring of closed caption data to the slave device for displaying captions is started as described, 
the process proceeds to step S113. In step S113, as in step S101, the CPU 129 determines whether the transport 
stream being supplied to the demultiplexer 124 includes more closed caption data to be displayed. 
[0431] If it is determined in step S1 13 that no closed caption data is included, the process jumps to step S1 15 without 
performing stepS114. In step S1 14, the CPU 129 controls thelEEE1394 interface 133 so as to stop the transfer of the 
closed caption data. Thus, the process exits the closed caption processing routine. 

[0432] On the other hand, if it is determined in step S1 1 3 that the transport stream being supplied to the demultiplexer 
124 includes more closed caption data to be displayed, the process proceeds to^tep S114. In step S114, the CPU 
129 determines whether a command to terminate displaying the closed caption (a closed caption display terminate 
command) has been received. 

[0433] If it is determined in step S114 that the closed caption display terminate command has not been received, 
the processing flow returns to step S113 to repeat the process described above. That is, in this case, transferring of 
closed caption data to the slave device for displaying captions is continued. 

[0434] On the other hand, if it is determined in step S114 that the closed caption display terminate command has 
been received, that is, if the IR receiver 135 has received an infrared ray, corresponding to the closed caption display 
terminate command, emitted from the remote commander 15 in response to a turning-off operation, performed by a 
user, on the guide button switch 63 of the remote commander 15 (Fig. 7) (or the guide button switch 93 of the remote 
commander 35 (Fig. 8)), the process proceeds to step S115. In step S115, the CPU 129 controls the IEEE1394 interface 
133 so as to stop the transfer of the closed caption data. Thus, the process exits the closed caption processing routine. 
[0435] If the closed caption processing shown in Fig. 34 is performed in the master device 1 , and rf, as a result, the 
closed caption display command is transmitted in step S110 and received by the slave device 2 specified as the caption 
displaying slave device (received by the IEEE1394 interface 153 of the slave device 2 (Fig. 11) and transferred to the 
CPU 149), the slave device 2 performs closed caption processing in accordance with an algorithm shown in a flow 
chart of Fig. 35. 

[0436] That is, in the slave device 2 (Fig. 11) serving as a slave device for displaying captions, first in step S121 , the 
IE EE 1394 receives coefficient seed data for use in dealing with closed captions transmitted in step S110 in Fig. 34, 
together with the dosed caption display command, by the master device 1 . The received coeffident seed data for use 
in dealing with dosed captions and the dosed caption display command are transferred to the CPU 149. The process 
then proceeds to step S122. 

[0437] In step S122, the CPU 149 transfers the coeffident seed data for use in dealing with closed captions to the 
signal processor 157, which sets (stores) the received coeffident seed data for use in dealing with closed captions in 
the coeffident seed data memory 207 (Fig. 29). In the above process, coeffident seed data existing in the coeffident 
seed data memory 207 is transferred to the EE PROM 1 57 B and saved therein before the coefficient seed data for use 
in dealing with dosed captions is stored in the coefficient seed data memory 207. 

[0438] In a case in which the slave device 2 serving as the slave device for displaying captions has caption coefficient 
seed data, that is, if coefficient seed data for use in dealing with dosed caption is initially stored in the coeffident 
memory 207 in the signal processor 157 of the slave device 2, steps S121 and S122 and also step S128 which will be 
described later may be skipped. 

[0439] The process then proceeds to step S123. In step S123, the CPU 149 determines whether the external input 
select command transmitted in step S111 in Fig. 34 by the master device 1 has been received. If it is determined that 
the command has not been received, the process returns to step S123. 

[0440] However, rf it is determined in step S123 that the external input select command transmitted from the master 
device 1 has been received, that is, if the external input select command transmitted from the master device 1 has 
been received by the IEEE1394 interface 153 and then transferred to the CPU 149, the process proceeds to step S124. 
In step S1 24, the CPU 149 makes selection of input so that the dosed caption data received by the 1EEE1 394 interface 
153 is supplied to the signal processor 157. The process then proceeds to step S125. 
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[0441] In step S125, the CPU 149 determines whether dosed caption data, whose transmission from the master 
device 1 is started in step S112 in Fig. 34, has been received. 

[0442] On the other hand, if it is determined in step S1 25 that the closed caption display terminate command trans- 
mitted from the master device 1 has been received, that is, if the closed caption data transmitted from the master 

5 device 1 has been received by the IEEE 1 394 interface 1 53 and then transferred to the CPU 1 49, the process proceeds 
to step S126. In step S126, the CPU 149 supplies the dosed caption data to the signal processor 157. The signal 
processor 157 performs video data conversion on the received closed caption data, using the coefficient seed data for 
use in dealing with closed caption, stored in step S122 in the coefficient seed data memory 207 (Fig. 29). 
[0443] More spedfically, in this case, the signal processor 1 57 decodes the closed caption data received from the 

10 CPU 149 and performs video data conversion on the dosed-caption video data obtained via the decoding process, 
using tap coefficients generated from the coefficient seed data for use in dealing with dosed captions, stored in the 
coefficient seed data memory 207, so as to convert the closed-caption data into high-resolution form. 
[0444] In step S127, the high-resolution closed-caption video data is supplied to the CRT 31 via the frame memory 
147 and the NTSC encoder 148 and displayed on the CRT 31. The process then returns to step S125, and steps S125 

15 to S127 are performed repeatedly until it is determined in step S125 that no further closed caption data is received 
from the master device. 

[0445] If it is determined in step S125 that no further dosed caption data is received from the master device 1 , that 
is, if the IEEE1394 interface 153 cannot receive further dosed caption data, the process proceeds to step S128. In 
step S128, the signal processor 157 resets (by means of overwriting) the original coefficient seed data saved in the 
20 EE PROM 1 57 B into the coeffident seed data memory 207 (Fig. 29). The process then exits the closed caption process- 
ing routine. 

[0446] Thus, in the dosed caption processing routine performed by the master device according to the flow shown 
in Fig. 34 and the dosed caption processing routine performed by the slave device according to the flow shown in Fig. 
35, when any television set in the scalable TV system does not have coefficient seed data for use in dealing with dosed 
25 captions, the master device 1 displays an image in such a manner that video data of a given television broadcast 
program and video data of a dosed caption superimposed on the former video data are displayed on the CRT 11 in a 
similar manner to a conventional television set having a built-in closed caption decoder. 

[0447] On the other hand, in the case in which some television set in the scalable TV system has coefficient seed 
data for use in dealing with dosed captions, only video data of a television broadcast program is displayed on the CRT 
30 11 of the master device 1. In this case, closed-caption video data corresponding to the video data displayed on the 
CRT 11 of the master device 1 is converted into high-resolution video data and displayed on the CRT 31 of the slave 
device 2 serving as the slave device for displaying captions. 

[0448] This allows a user to view the video data of television broadcast programs without being disturbed by the 
video data of dosed captions. Furthermore, the user can view video data of dosed captions with the high resolution. 

35 [0449] Even in the case in which any television set in the scalable TV system does not have coefficient seed data 
for use in dealing with closed captions, the video data of closed captions can be displayed on the CRT 31 of the slave 
device 2 serving as the slave device for displaying captions separately from the video data of television broadcast 
programs. In this case, although the video data of dosed captions displayed does not have a high resolution, the user 
can view video data of television broadcast programs without being disturbed by video data of dosed captions. 

40 [0450] Although in the example described above, closed-caption video data is displayed only one slave device 2 
spedfied as the slave device for displaying captions, closed-caption video data may be displayed on two or more slave 
devices in the scalable TV system. For example, when there are two or more dosed caption data corresponding to 
two or more languages, dosed-caption video data of respective languages may be displayed separately on different 
slave devices. 

45 [0451] The scalable TV system has a special capability of displaying a part of video data in an enlarged fashion. 
This spedal capability is achieved by a partial-image enlarging process performed by the master device 1 and the 
slave device 2. 

[0452] A command to perform the partial-image enlarging process may be issued, for example, via a menu screen. 
[0453] More spedfically, if a user operates the menu button switch 54 of the remote commander 15 (Fig. 7) (or the 

50 menu button switch 84 of the remote commander 35 (Fig. 8)), a menu screen is displayed on the CRT 1 1 of the master 
device 1 (or the CRT 31 of the slave device 2). An icon indicating the partially enlarge command (hereinafter, referred 
to as a partially enlarge icon) is displayed on the menu screen. If a user clicks on the partially enlarge icon by operating 
the remote commander 15, the partial-image enlarging process is started in the master device 1 and the slave device 2. 
[0454] Referring to a flow chart shown in Fig. 36, the partial-image enlarging process performed by the master device 

55 js described below. 

[0455] When video data of a television broadcast program (hereinafter, referred to as program video data) is being 
displayed on the CRT 11 of the master device 1, if the partially enlarge icon is dicked, the following process is started. 
That is, first, in step S131, the CPU 129 selects a slave device for displaying the full image of program video data 
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(hereinafter, referred to as a full-image displaying slave device) which is currently being displayed on the CRT 11 of 
the master device 1, from television sets in the scalable TV system. The process then proceeds to step S132. 
[0456] Herein, the CPU 129 may select only one television set or two or more television sets serving as slave devices 
(or all television sets serving as slave devices) as full-image displaying slave devices from slave devices in the scalable 
TV system. 

[0457] In step S132, the CPU 129 communicates with the full-image displaying slave device via the IEEE1394 in- 
terface 133 to determine whether the electric power of the full-image displaying slave device is in an on-state. 
[0458] If it is determined in step S1 32 that the electric power of the full-image displaying slave device is not in the 
on-state, the process proceeds to step S133. In step S133, the CPU 129 transmits, via the IEEE1394 interface 133, 
a power-on command to the full-image displaying slave device to turn on the power of the full-image displaying slave 
device. The process then proceeds to step S134. 

[0459] On the other hand, if it is determined in step S132 that the full-image displaying slave device is in the on- 
state, the process jumps to step S134 without performing step S133. In step S134, the CPU 129 controls the signal 
processor 1 37 so as to display a message on the CRT 1 1 , for example, in the OSD fashion to prompt a user to specify 
which part of the image being displayed on the CRT 11 should be enlarged (hereinafter, such a message will be referred 
to as a message requesting specifying a part to be enlarged). 

[0460] That is, in this case, under the control of the CPU 129, the signal processor 137 generates OSD data of the 
message requesting specifying a part to be enlarged and superimposes it on the program video data stored in the 
frame memory 127. The program video data including the superimposed OSD data of the message requesting spec- 
ifying a part to be enlarged is supplied from the frame memory 127 to the CRT 11 via the NTSC encoder 128. The CRT 
11 displays, in the OSD manner, the message requesting specifying a part to be enlarged, together with the program 
video data. 

[0461] In the next step S135, the CPU 129 determines whether the user has specified a part to be enlarged in 
response to the message requesting specifying a part to be enlarged. If a part to be enlarged has not been specified, 
the process returns to step S135. 

[0462] However, if it is determined in step S135 that the user has specified a part to be enlarged, that is, if the IR 
receiver 1 35 has received an infrared ray transmitted in response to an operation performed by the user on the remote 
commander 15 (or the remote commander 35) to specify a part of an image displayed on the display screen of the 
CRT 11 , and if the signal corresponding to the infrared ray has been supplied to the CPU 129, the CPU 129 determines 
that the part to be enlarged has been specified. The process then proceeds to step S136. 

[0463] In step S136, the CPU 129 transmits, via the IEEE1394 interface 133, an external input select command to 
the full-image displaying slave device to command it to select the input applied to the IEEE1394 interface 153 (Fig. 
11) of the full-image displaying slave device and display the selected input on the CRT 31 . The process then proceeds 
to step S1 37. 

[0464] In step S1 37, the CPU 129 starts transferring the program video data to the full-image displaying slave device. 
[0465] More specifically, the CPU 129 requests the demultiplexer 124 to supply TS packets included in the transport 
streams and being supplied to the MPEG video decoder 125. In response to the request, the demultiplexer 124 supplies 
the TS packets to the CPU 129. Furthermore, the CPU 129 transfers, via the IEEE1394 interface 133, the TS packets 
received from the demultiplexer 124 to the full-image displaying slave device. Thus, TS packets corresponding to the 
program video data displayed on the CRT 11 of the master device 1 are transferred to the full-image displaying slave 
device, which performs the partial-image enlarging process as will be described later with reference to Fig. 37, thereby 
displaying the program video data corresponding to the TS packets. That is, the full image of the program video data 
previously displayed on the master device 1 is displayed on the full-image displaying slave device. 
[0466] Alternatively, the CPU 1 29 may read, via the signal processor 1 37, MPEG-decoded program video data from 
the frame memory 127 and may transfer it, instead of the TS packets, to the full-image displaying slave device. In this 
case, the full-image displaying slave device can display the program video data without having to perform MPEG- 
decoding. 

[0467] After starting the transferring of the TS packets to the fulWmage displaying slave device, the process proceeds 
to step S138. In step S138. the CPU 129 controls the signal processor 137 so as to perform video data conversion on 
the program video data stored in the frame memory 1 27 such that a specified area of the program video data is enlarged 
using coefficient seed data for use in resizing stored in the coefficient seed data memory 167 (Fig. 22). 
[0468] That is, in the present embodiment, the coefficient seed data memory 167 in the signal processor 137 (Fig. 
22) of the master device 1 stores at least the coefficient seed data for use in resizing, and the signal processor 137 
performs the video data conversion on the specified enlarging area of the program video data stored in the frame 
memory 127 so that the enlarging area of the program video data is enlarged (resized) by a specified factor, using tap 
coefficients generated from the coefficient seed data for use in resizing stored in the coefficient seed data memory 
167, thereby producing partial enlarged video data. 

[0469] In step S138, the resultant partial enlarged video data is supplied via the frame memory 127 and the NTSC 
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encoder 128 to the CRT 11 and displayed thereon. 

[0470] That is, in this case, an enlarging area, centered at an enlarging point specified by a user, of the program 
video data is enlarged and resultant partial enlarged video data is displayed on the CRT 11 of the master device 1. 
[0471] The enlarging area may be specified, for example, by an enlargement factor. 
5 [0472] In the partially enlarging process, a default value of the enlargement factor (default enlargement factor) is 
preset. The CPU 129 sets a parameter corresponding to the default enlargement factor into the parameter memory 
168 in the signal processor 137 (Fig. 22). The signal processor 137 performs resizing of given program video data 
according to the default enlargement factor. 

[0473] On the other hand, the size of image displayed on the CRT 11, that is, the size of the display screen has a 
10 predetermined value. 

[0474] Thus, the CPU 1 29 calculates the enlarging area so that if the enlarging area, centered at a specified enlarging 
point, of given video data is enlarged by the default enlargement factor, the size of resultant enlarged video data has 
a size equal to the size of the display screen of the CRT 11. 

[0475] The enlargement factor employed in the video data conversion performed in step S1 38 may be specified by 
15 a user. 

[0476] For example, the CPU 129 controls the signal processor 137 so as to display, on the CRT 11, a lever which 
can be operated by the user by operating the remote commander 15 (or the remote commander 35) to specify the 
enlargement factor (hereinafter, such a lever displayed on the CRT 11 will be referred to as an enlargement factor 
specifying lever). The enlargement factor may be specified by the position of the enlargement factor specifying lever 

20 [0477] In this technique, if the user moves the position of the enlargement factor specifying lever by operating the 
remote commander 15, the CPU 129 detects the position to which the lever has been moved, and the CPU 129 sets 
the parameter indicating the enlargement factor corresponding to the detected position into the parameter memory 
168 in the signal processor 137 (Fig. 22). Furthermore, the CPU 129 specifies the enlarging area centered at the 
enlarging point in accordance with the enlargement factor corresponding to the position of the enlargement factor 

25 specifying lever in a similar manner as in the above-described case in which the default enlargement factor is employed. 
The CPU 129 then commands the signal processor 137 to perform video data conversion (resizing) on the specified 
enlarging area of given program video data. 

[0478] Thus, a partial enlarged video data obtained by enlarging the enlarging area, centered at the enlarging point, 
of the given program video data by an enlargement factor specified by the user by operating the remote commander 

30 15 is displayed on the CRT 11 . 

[0479] The enlargement factor specifying lever may be displayed, in the OSD fashion, on the CRT 11 of the master 
device 1 or on another television set other than the master device 1 in the scalable TV system. 
[0480] The process then proceeds to step S140. In step S140, the CPU 129 determines whether a command to 
terminate displaying the partial enlarged video data (hereinafter, referred to as a partial enlargement terminate com- 

35 mand) has been received. 

[0481] If it is determined in step S140 that the partial enlargement terminate command has not been received, the 
processing flow returns to step S1 33 to repeat the process described above. 

[0482] On the other hand, if it is determined in step S140 that the partial-image enlarging terminate command has 
been received, that is, if a user operates the remote commander 15 (Fig. 7) so as to display a menu screen on the 
40 CRT 11 and re-click on the partially enlarge icon on the menu screen thereby causing an infrared ray corresponding 
to the partially enlarge command to be emitted from the remote commander 15 and received by the IR receiver 135 
and finally transferred to the CPU 129, the process proceeds to step S141. In step S141, the CPU 129 controls the 
IEEE 1394 interface 133 so as to terminate the transferring of program video data to the full-image displaying slave 
device. 

45 [0483] The process then proceeds to step S142. In step S142, the CPU 129 controls the signal processor 1 37 so 
as to stop the resizing process. Thus, the processing flow exits the partially enlarging routine. Thereafter, an image 
with a normal size is displayed on the CRT 11. 

[0484] Now, referring to a flow chart shown in Fig. 37, a partially enlarging process performed by a slave device 
specified as operating as a full-image displaying slave device is described below. 
50 [0485] In the slave device 2 serving as the full-image displaying slave device, first, in step S151 , the CPU 149 de- 
termines whether an external input select command transmitted, in step S136 in Fig. 36, by the master device 1 has 
been received. If the command has not been received, the process returns to step S151. 

[0486] If it is determined in step S151 that the external input select command transmitted from the master device 1 
has been received, that is, if the external input select command transmitted from the master device 1 has been received 
55 by the IEEE 1394 interface 1 53 and then transferred to the CPU 149, the process proceeds to step S1 52. In step S1 52, 
the CPU 149 makes selection of input so that program video data received by the IEEE1 394 interface 153 is supplied 
to the MEPG video decoder 145 via the demultiplexer 144. The process then proceeds to step S153. 
[0487] In step S153, the CPU 149 determines whether program video data, whose transmission from the master 
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device 1 is started in step S1 37 in Fig. 36, has been received. 

[0488] On the other hand, if it is determined in step S153 that the video program data transmitted from the master 
device 1 has been received, that is, if the program video data transmitted from the master device 1 has been received 
by the IEEE1394 interface 153 and then transferred to the CPU 149, the process proceeds to step S154. In step S154, 
the CPU 149 displays the received program vide data on the CRT 31. 

[0489] More specifically, in the present embodiment, in step S137 in Fig. 36, the master device 1 starts transmission 
of program video data in the form of TS packets to the slave device 2 serving as the full-image displaying slave device. 
After transmission of the program video data is started, the CPU 1 49 supplies the TS packets received from the master 
device 1 via the IEEE1394 interface 153 to the MPEG video decoder 145 via the demultiplexer 144. The MPEG video 
decoder 145 performs MPEG-decoding on the TS packets thereby acquiring program video data. The resultant program 
video data is stored into the frame memory 147. The program video data is then supplied from the frame memory 147 
to the CRT 31 via the NTSC encoder 148. 

[0490] The process then returns to step S153, and steps S153 and to S154 are performed repeatedly until it is 
determined in step S153 that no further program video data is received from the master device 1. 
[0491] If it is determined in step S153 that program video data is not received from the master device 1, that is, if 
the IEEE 1394 interface 153 cannot receive further program video data, the partially enlarging process is ended. 
[0492] In the partial-image enlarging process performed by the master device according to the flow shown in Fig. 
36 and the partial-image enlarging process performed by the slave device according to the flow shown in Fig. 37, for 
example, when program video data is being displayed on the master device 1 located in the second row and in the 
second column in the arrangement of the scalable TV system as shown in Fig. 38A, if a certain point P in the program 
video data is specified as the enlarging point, a rectangular area (represented by a broken line in Fig. 38A) centered 
at the enlarging point P (the center of gravity of the rectangular area) is set as the enlarging area EA, and an enlarged 
partial image obtained by enlarging the program video data in the enlarging area EA is displayed, instead of the program 
video data, on the master device 1 as shown in Fig. 38 B. 

[0493] Furthermore, for example, in the case in which a slave device 2 21 located on the left side of the master device 
1 is selected as the full-image displaying slave device, the full image of the program video data initially displayed on 
the master device 1 is displayed on the slave device 2 21 specified as the full-image displaying slave device. 
[0494] This allows the user to view details of a desired part of program video data on the master device 1 . The user 
can also view the whole image of the program video data on the slave device 2. Furthermore, in the present embodiment, 
as described above, the user is allowed to specify the enlargement factor in the partially enlarging of the video data 
by operating the remote commander 15, and thus the user can view the details of a desired part of the program video 
data by enlarging it by an arbitrary desired factor. 

[0495] In the signal processor 137 (Fig. 22) of the master device 1 (Fig. 10), video data conversion is performed 
such that the program video data in the enlarging area is converted into enlarged partial video data, using the tap 
coefficients w n produced from the coefficient seed data in accordance with equation (1). When only equation (1) is 
viewed, the video data conversion seems as if it were performed by means of simple interpolation. However, in reality, 
the tap coefficients w n in equation (1) are produced on the basis of coefficient seed data which is obtained via learning 
using teacher data and student data as described earlier with reference to Figs. 24 to 28, and thus the video data 
conversion using the tap coefficients w n produced such coefficient seed data allows components included in the teacher 
data to be reproduced. More specifically, for example, when coefficient seed data for use in resizing is used, details, 
which are not included in the original image, are reproduced in an enlarged image obtained using tap coefficients w n 
produced on the basis of the coefficient seed data. This means that the resizing by means of the video data conversion 
according to equation (1) using the coefficient seed data obtained via learning is quite different from the enlarging 
process by means of simple interpolation. 

[0496] However, enlargement of program video data in a specified enlarging area to an enlarged partial video data 
may be performed by means of simple interpolation without using the tap coefficients produced on the basis of the 
coefficient seed data. In this case, details which are not included in the original program video are not reproduced in 
the resultant enlarged image, and the enlarged image becomes blurred and a block-like pattern appears, as the en- 
largement factor increases. 

[0497] Although in the present embodiment, a partial enlarged image of video data is displayed on the master device 
1 and a full image of program video data is displayed on the slave device 2, the partial enlarged image may be displayed 
on the slave device 2 while the program video data is still displayed on the master device 1. 

[0498] Although in the present embodiment, a partial enlarged image of video data is displayed on the master device 
1 and a full image of program video data is displayed on a slave device 2 specified as the full-image displaying slave 
device, in addition to those images, an enlarged partial image or a full image of program video data may be displayed 
on another television set in the scalable TV system. 

[0499] In the scalable TV system, the full image may be displayed on the master device 1 , while partial video data 
enlarged by various different factors may be displayed on other television sets serving as slave devices 2 U to 233. In 
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this case, partial video data enlarged by different factors may all be produced by the signal processor 1 37 of the master 
device 1 and supplied to the respective television sets serving as slave device 2 1t to 233, or partial enlarged video data 
may be produced by the signal processor 157 of each of the television sets serving as slave device 2 ln to 2 33 . 
[0500] In the present embodiment, the coefficient seed data for use in resizing is assumed to be stored in the master 
5 device 1 . However, when the coefficient seed data for use in resizing is not stored in the master device 1 , if the coefficient 
seed data for use in resizing is stored in another television set in the scalable TV system, the master device 1 may 
acquire the coefficient seed data for use in resizing from that television set. The coefficient seed data for use in resizing 
may also be acquired from a coefficient seed data server 

[0501] Although in the above-described example, resizing of program video data is performed such that the image 
10 size is increased, resizing may also be performed such that the image size is reduced. 

[0502] Although in the above-described example, video data of a television broadcast program (program video data) 
is enlarged, the partially enlarging process may also be performed on other video data such as a that supplied from 
an external device (such as an optical-disk storage device, a magnetooptical-disk storage device, or a VTR). 
[0503] Furthermore, partially enlarging may be performed not only in such a manner that a part of program video 
15 data is enlarged by the same factor in both horizontal and vertical directions but also in such a manner that a part of 
program video data is enlarged by different factors in horizontal and vertical directions. 

[0504] Although in the present embodiment, only a part of program video data is enlarged such that the resultant 
enlarged image can be displayed on the display screen of the CRT 11 , enlargement may also be performed such that 
the whole image is enlarged. In this case, only a part of the resultant enlarged image is displayed but the whole enlarged 

20 image cannot be displayed on a single : CRT 11. However, a user may change the part displayed on the CRT 11 by 
operating the remote commander 15 so that a desired part of the program video data is displayed. 
[0505] In addition to the capability of enlarging a part of video data, the scalable TV system also has a special 
capability of enlarging the full image. This special capability is achieved by a full-image enlarging process performed 
by the master device 1 and slave devices 2. 

25 [0506] A full-image enlarge command can be issued via the menu screen, as in the case of the partially enlarge 
command. 

[0507] More specifically, if a user operates the menu button switch 54 of the remote commander 15 (Fig. 7) (or the 
menu button switch 84 of the remote commander 35 (Fig. 8)), a menu screen is displayed on the CRT 11 of the master 
device 1 (or the CRT 31 of the slave device 2). An icon indicating the full image enlarge command (hereinafter, referred 
30 to as a full image enlarge icon) is displayed on the menu screen. If a user clicks on the full image enlarge icon by 
operating the remote commander 1 5, the full image enlarging process is started in the master device 1 and the slave 
device 2. 

[0508] Referring to a flow chart shown in Fig. 39, the full image enlarging process performed by the master device 
is described below. 

35 [0509] When video data of a television broadcast program (program video data) is being displayed on the CRT 11 
of the master device 1, if the full image enlarge icon is clicked, the following process is started. That is, first, in step 
S161, the CPU 129 of the master device 1 (Fig. 10) transmits, via the IEEE1394 interface 133, coefficient seed data 
for use in resizing to all slave devices in the scalable TV system. 

[051 0] In the present embodiment, it is assumed that coefficient seed data for use in resizing is stored in the coefficient 
40 seed data memory 167 in the signal processor 137 (Fig. 22) of the master device 1 . Thus, in step S161 , the CPU 129 
reads the coefficient seed data for use in resizing from the signal processor 137 and transmits it. 
[0511] In the case in which the coefficient seed data for use in resizing is not stored in the master device 1, the 
coefficient seed data for use in resizing may be acquired from another television set, having the coefficient see data 
for use in resizing, in the scalable TV system or from a coefficient seed data server, as in the case of the partially 
45 enlarging process. 

[0512] The process then proceeds to step S162. In step S162, the CPU 129 communicates with all slave device 2 11 
to 233 in the scalable TV system via the IEEE1 394 interface 133 to determine whether there is a slave device 2y which 
is in a power-off state. 

[0513] If it is determined in step S162 that there is a slave device 2 l} which is in the power-off state, the process 
50 proceeds to step S163. In step S 163, the CPU 129 transmits, via the IE EE 1394 interface 133, a power-on command 
to the slave device 2y thereby turning on the power of the slave device 2jj. The process then proceeds to step S164. 
[0514] However, if it is determined in step S162 that there is no slave device 2^ which is in the power-off state, the 
process jumps to step S164 without performing step S163. In step S164, the CPU 129 transmits, via the IE EE 1394 
interface 1 33, an external input select command to all slave devices 2^ to 233 to command them to select data input 
55 to the IEEE 1394 interface 153 (Fig. 11) thereof and display the input data on the CRT 31. The process then proceeds 
to step S165. 

[051 5] In step S 165, the CPU 1 29 initializes the enlargement factor N, by which program video data is to be enlarged, 
to a value of 1. The CPU 129 further sets the maximum enlargement factor N max and the enlargement pitch a. 
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[051 6J In the full image enlarging process performed in the scalable TV system including, for example, 3x3 television 
sets as shown in Fig. 1 A, the full image of program video data currently displayed on the master device 1 is gradually 
enlarged over the screens of the slave devices 2^ to 233 while a central part of the full image is displayed on the screen 
of the master device 1 , until the enlarged full image of the program video data is displayed over the total of display 
screens of 3 x 3 television sets. 

[0517] Thus, the full image of program video data initially displayed on the master device 1 is finally enlarged to a 
size equal to the total size of screens including all screens of television sets of the scalable TV system. The ratio of 
the size of the final enlarged full image to the size of the initial program video data (initially displayed on the master 
device 1) is set as the maximum enlargement factor N max . That is, in the present embodiment, the full image of program 
video data initially displayed on the master device 1 is enlarged into the full image with the size equal to the total screen 
size including display screens of 3 x 3 television sets, and thus the image is enlarged by a factor of 3 as can be 
understood by a simple calculation of the enlargement factor in terms of, for example, a diagonal line. Therefore, the 
maximum enlargement factor N,^ is set to 3. 

[0518] In the full image enlarging process, as described above, the initial full image of program video data displayed 
on the master device 1 is gradually enlarged. This can be achieved, for example, the program video data is enlarged 
while gradually increasing the enlargement factor N until the enlargement factor N becomes equal to the maximum 
enlargement factor N,^. In the above process, when the enlargement factor N is gradually increased from 1 to N^, 
increasing is performed at enlargement pitches a. For example, the enlargement pitch a is set to a value obtained by 
dividing (N^ - 1 ) by a particular value greater than 1 (hereinafter, the particular value will be referred to as the number 
of times enlargement is performed), 

[0519] The number of times enlargement is performed may be set, in advance, in the master device 1 or a user may 
set it by operating the remote commander 1 5 (or the remote commander 35). When the number of times enlargement 
is performed is set to a small value, the initial program video data displayed on the master device 1 is quickly enlarged 
to a final enlarged full image. In contrast, when the number of times enlargement is performed is set to a large value, 
the initial program video data displayed on the master device 1 is gradually enlarged to a final enlarged full image. 
[0520] After completion of initializing the enlargement factor N and setting the maximum enlargement factor 
and the enlargement pitch a in step S165, the process proceeds to step S166. In step S166, the CPU 129 resets the 
enlargement factor N to N + a. The process then proceeds to step S167. 

[0521] If the new enlargement factor N set in step S166 is greater than the maximum enlargement factor N^, the 
CPU 129 sets the enlargement factor N to the maximum enlargement factor N^. 

[0522] In step S167, the CPU 129 determines an enlarging area of the program video data initially displayed on the 
master device 1, which is to be enlarged by the signal processor 137 of the master device 1 and also determines 
enlarging areas to be enlarged by the signal processors 157 of the respective slave devices 2y (Fig. 11), on the basis 
of the enlargement factor N set in step S165. The process then proceeds to step S168. In step S168, the CPU 129 
determines displaying areas of the enlarged program video data, to be displayed on the CRT 11 of the master device 

I and CRTs 31 of the respective slave devices 2^ (Fig. 11) (each enlarged video data displayed on each CRT will also 
be referred to as partial enlarged video data) on the basis of the enlargement factor N set in step S165. The process 
then proceeds to step S1 69. 

[0523] Now, referring to Fig. 40, the method of calculating the enlarging area associated with the master device 1 
(the enlarging area to be enlarged by the signal processor 137 of the master device 1), the enlarging areas associated 
with slave devices 2|j (the enlarging areas to be enlarged by the signal processors 157 of slave devices 2^), and the 
displaying area associated with slave devices 2y (areas of respective partial enlarged video data produced by enlarging 
the enlarging areas of the program video data, to be displayed on CRTs 31 of the respective slave devices 2jj) on the 
basis of the enlargement factor N is described below. 

[0524] Fig. 40A shows the total display screen including display screens of 3 x 3 television sets of the scalable TV 
system. 

[0525] More specifically, the total display screen of the scalable TV system includes the display screen of the CRT 

II of the master device 1 and display screens of CRTs 31 of eight slave devices 2„ to 2 33 . That is, the total display 
screen includes display screens of a total of nine CRTs. As described earlier, the screen sizes are identical for the 
master device 1 and all slave devices 2y. 

[0526] In the full image enlarging process in which, as described above, the full image of the program video data 
initially displayed on the master device 1 is gradually enlarged, the program video data initially displayed on the master 
device 1 is denoted by video data Q, and the enlarged full video data obtained by enlarging the video data Q by a 
factor of N is denoted by video data Q\ 

[0527] Herein, if the vertical screen size of the master device 1 and the horizontal screen size are denoted by a and 
b, respectively, the vertical and horizontal image sizes of the program video data Q are equal to a and b, respectively. 
[0528] Because the enlarged full video data Q* is obtained by enlarging the program video data Q by the factor of N 
in both vertical and horizontal directions, the vertical and horizontal sizes of the enlarged full video data Q are Na and 
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Nb, respectively. 

[0529] In the full image enlarging process, as described above, the enlarged full video data Q* obtained by enlarging 
the full program video data Q initially displayed on the master device 1 is displayed such that the central part thereof 
is displayed on the master device 1. In order to display the enlarged full video data Q* over the entire total screen 
5 including the display screens of the master device 1 and the slave devices 2^ to 233 such that the central part of the 
enlarged full video data Q' is displayed on the display screen of the master device 1, it is required that the master 
device 1 should display assigned partial enlarged video data in the area, denoted by R n in Fig. 40A, of the total display 
screen, and slave devices 2y should display assigned partial enlarged video data in the respective areas denoted by 
Rjj in Fig. 40A. 

10 [0530] To meet the above requirement, in step S 168 in Fig. 39, the area R 1 is determined as the displaying area of 
the master device 1 and the areas Ry are determined as the displaying areas of the slave devices 2y. 
[0531] That is, as for the master device 1 , the full screen area is employed as the displaying area R v In the case of 
a slave device 2^ at the upper left location relative to the location of the master device 1, a rectangular area with a 
size of ((Nb-b/2) x ((Na-a)/2) (horizontal length x vertical length) at the lower right corner of the display screen is 

15 employed as the displaying area. For a slave device 2 12 at the upper location relative to the location of the master 
device 1 , a rectangular area with a size of b x ((Na-a)/2) (horizontal length x vertical length) on the lower side of the 
display screen is employed as the displaying area R 12 - For a slave device 2 13 at the upper right location relative to the 
location of the master device 1, a rectangular area with a size of ((Nb-b/2) x ((Na-a)/2) (horizontal length x vertical 
length) at the lower left comer of the display screen is employed as the displaying area R 13 . For a slave device 2 21 on 

20 the left side of the master device 1 , a rectangular area with a size of ((Nb-b/2) x a (horizontal length x vertical length) 
on the right side of the display screen is employed as the displaying area R 21 . For a slave device 2 23 on the right side 
of the master device 1 , a rectangular area with a size of ((Nb-b/2) x a (horizontal length x vertical length) on the left 
side of the display screen is employed as the displaying area R^. For a slave device 2 31 at the lower left location 
relative to the location of the master device 1 , a rectangular area with a size of ((Nb-b/2) X ((Na-a)/2) (horizontal length 

25 x -vertical length) at the upper right comer of the display screen is employed as the displaying area R 31 . For a slave 
device 2 32 at the lower location relative to the location of the master device 1 , a rectangular area with a size of b x 
((Na-a)/2) (horizontal length x vertical length) on the upper side of the display screen is employed as the displaying 
area R 32 . For a slave device 2 33 at the lower right location relative to the location of the master device 1 , a rectangular 
area with a size of ((Nb-b/2) x (<Na-a)/2) (horizontal length x vertical length) at the upper left corner of the display 

30 screen is employed as the displaying area R 33 . 

[0532] Herein, if the total area including the displaying area R 1 of the master device 1 and the displaying areas Ry 
of the slave devices 2y shown in Fig. 40A is regarded as the area of the enlarged full image of video data Q\ the 
respective video data of the areas Rj and Ry should be partial enlarged video data enlarged from corresponding parts 
of the original program video data Q. Thus, it is required to determine respective enlarging areas of the program video 

35 data Q from which the corresponding partial enlarged video data, which are to be displayed in the displaying area R 1 
of the master device 1 and the displaying areas Ry of the slave devices 2 Yi , are enlarged. 

[0533] Thus, in step S167, as shown in Fig. 40B, areas r, and ry of the image of the original program video data Q 
corresponding to the areas R-, and Ry, respectively, of the enlarged full image of video data Q' are determined as the 
enlarging area associated with the master device 1 and the enlarging areas associated with the slave devices 2y, 
40 respectively. 

[0534] Because the enlarged full image of the video data Q' with a size of Nb X Na is obtained by enlarging, by a 
factor of N, the image of the program video data Q with a size of b x a, the areas r n and ry of the image of the program 
video data Q can be calculated by multiplying the areas R-, and Ry of the enlarged full image of the video data Q' by 
a factor of 1/N, and the calculated areas r 1 and ry can be employed as the enlarging areas associated with the master 

45 device 1 and the slave devices 2 Vy 

[0535] More specifically, a central area of the image of the program video data Q with a size of b/N x a/N (horizontal 
length x vertical length) is employed as the enlarging area r y associated with the master device 1. An area of the 
image of the program video data Q with a size of (b-b/N)/2 Zx (a-a/N)/2 (horizontal length x vertical length) is employed 
as the enlarging area r ni associated with the slave device 2^ at the upper left location relative to the location of the 

50 master device 1 . An area of the image of the program video data Q with a size of b/N x (a-a/N)/2 (horizontal length x 
vertical length) is employed as the enlarging area r 12 associated with the slave device 2 12 located on the upper side 
of the master device 1 . An area of the image of the program video data Q with a size of (b-b/N)/2 X (a-a/N)/2 (horizontal 
length x vertical length) is employed as the enlarging area r 13 associated with the slave device 2 13 at the upper right 
location relative to the location of the master device 1. An area of the image of the program video data Q with a size 

55 of (b-b/N)/2 x a/N) (horizontal length x vertical length) is employed as the enlarging area r 21 associated with the slave 
device 2 21 located on the left side of the master device 1. An area of the image of the program video data Q with a 
size of (b-b/N)/2 x a/N) (horizontal length x vertical length) is employed as the enlarging area r 23 associated with the 
slave device 2 23 located on the right side of the master device 1 . An area of the image of the program video data Q 
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with a size of (b-b/N)/2 x (a-a/N)/2 (horizontal length x vertical length) is employed as the enlarging area r 31 associated 
with the slave device 2 31 at the lower left location relative to the location of the master device 1. An area of the image 
of the program video data Q with a size of b/N x (a-a/N)/2 (horizontal length x vertical length) is employed as the 
enlarging area r 32 associated with the slave device 2 32 located on the lower side of the master device 1. An area of 
the image of the program video data Q with a size of (b-b/N)/2 x (a-a/N)/2 (horizontal length x vertical length) is 
employed as the enlarging area r^ associated with the slave device 2^ at the lower right location relative to the location 
of the master device 1 . 

[0536J Referring again to Fig. 39, in step S169, the CPU 129 transmits, via the IEEE1394 interface 133, an enlarge- 
and-display command together with program video data, the enlargement factor N, the enlarging areas, and the dis- 
playing areas to the respective slave devices 2 Vj to request them to enlarge (resize) the video data using the coefficient 
seed data transmitted in step S161 and display the enlarged video data. 

[0537] 1" the above process, the CPU 129 acquires the program video data by requesting the demultiplexer 124 to 
supply TS packets included in the transport stream being supplied to the MPEG video decoder 125. In response to 
the request the demultiplexer 124 transmits the TS packets to the CPU 129. The CPU 129 transmits the received TS 
packets to the respective slave devices 2jj. 

[0538] As for the enlarging areas and displaying areas, the CPU 129 transmits the enlarging areas and displaying 
areas determined for respective slave devices 2y to the corresponding slave devices 2y. 

[0539] Instead of transmitting TS packets, the CPU 1 29 may transmit MPEG-decoded program video data, read via 
the signal processor 137 from the frame'memory 127, to the respective slave devices 2y. This makes unnecessary for 
each slave device 2^ to perform MPEG-decoding on the program video data. 

[0540] In the case in which MPEG-decoded program video data is transmitted to slave devices 2jj, part of the program 
video data corresponding to the enlarging area assigned to each slave device 2jj may be transmitted instead of trans- 
mitting the whole program video data. 

[0541] The process then proceeds to step S170. In step S170, the CPU 129 sets the parameter z corresponding to 
the enlargement factor N, determined in step S166, into the parameter memory 168 of the signal processor 137 (Fig. 
22). The process then proceeds to step S171. 

[0542] In step S171 , the CPU 129 controls the signal processor 137 (Fig. 22) so as to perform video data conversion 
on the program video data which is stored in the frame memory 127 and which is the same as that transmitted, in step 
S169, to the respective slave devices 2y so that the enlarging area r, (Fig. 40B) assigned to the master device 1 is 
enlarged. 

[0543] More specifically, in the present embodiment, the coefficient seed data for use in resizing is stored in the 
coefficient seed data memory 167 in the signal processor 1 37 (Fig. 22) of the master device 1, and the signal processor 
137 performs video data conversion on the enlarging area r, of the program video data stored in the frame memory 
127, using the coefficient seed data for use in resizing stored in the coefficient seed data memory 167 and using the 
tap coefficient generated from the parameter z stored in the parameter memory 168, so as to convert the enlarging 
area r, of the program video data into a partial enlarged video data (enlarged (resized) by the enlargement factor of N). 
[0544] In the above process, the CPU 129 controls the signal processor 137 so that the resultant partial enlarged 
video data is stored in the displaying area R t (Fig. 40A), assigned to the master device 1 , of the display screen of the 
CRT 11. That is, the signal processor 137 adjusts the displaying position so that the partial enlarged video data is 
displayed in the displaying area R n (Fig. 40A), assigned to the master device 1, on the display screen of the CRT 11. 
[0545] Note that in the case of the master device 1, as described earlier with reference to Fig. 40, the displaying 
area R 1 is identical to the size of the display screen of the CRT 11 , and thus, in practice, the adjustment of the displaying 
position is not necessary. 

[0546] In step S172, the signal processor 137 supplies the partial enlarged video data obtained in step S171 to the 
CRT 11 via the frame memory 127 and the NTSC encoder 128. The CRT 11 displays the received partial enlarged 
video data. 

[0547] Thus, in this case, the partial enlarged video data obtained by enlarging the enlarging area ^ of the initial 
program video data by the enlargement factor N is displayed over the whole area of the display screen of the CRT 1 1 
of the master device 1 . 

[0548] The process then proceeds to step S173. In step S173, the CPU 129 determines whether the enlargement 
factor N is smaller than the maximum enlargement factor N^. If it is determined in step S173 that the enlargement 
factor N is smaller than the maximum enlargement factor N,^, the processing flow returns to step S166 to repeat the 
process described above. 

[0549] On the other hand, if it is determined in step S173 that the enlargement factor N is not smaller than the 
maximum enlargement factor N,^, that is, in the case in which the enlargement factor N has been set, in step S166, 
to the maximum enlargement factor N^, the process proceeds to step S174. In step S174, as in step S169, the CPU 
129 transmits, via the IEEE1394 interface 133, the enlarge-and-display command together with the program video 
data, the enlargement factor N, the enlarging area, and the displaying area to each slave device 2g. The process then 
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proceeds to step S1 75. 

[0550] In step S1 75, the CPU 1 29 controls the signal processor 1 37 (Fig. 22) so as to perform video data conversion 
on the program video data which is stored in the frame memory 127 and which is the same as that transmitted, in step 
S174, to the respective slave devices 2y so that the enlarging area r 1 (Fig. 40B) assigned to the master device 1 is 
5 enlarged. 

[0551] More specifically, in this step S175, as in step S169, the signal processor 137 performs video data conversion 
on the enlarging area r n of the program video data stored in the frame memory 127, using the coefficient seed data for 
use in resizing stored in the coefficient seed data memory 167 and using the tap coefficient generated from the pa- 
rameter z stored in the parameter memory 168, so as to convert the enlarging area r t of the program video data into 
10 a partial enlarged video data (enlarged (resized) by the enlargement factor of N). 

[0552] In step S176, as in step S172. the resultant partial enlarged video data is supplied via the frame memory 127 
and the NTSC encoder 128 to the CRT 11 and is displayed thereon. 

[0553] Herein, the enlargement factor N. the enlarging areas, and the displaying areas transmitted in step S174 to 
the respective slave devices 2^ are those determined in steps S166 to S168, and thus the enlargement factor N is 
15 equal to the maximum enlargement factor N^, and the enlarging areas and the displaying areas are those determined 
on the basis of the enlargement factor N equal to the maximum enlargement factor N^. 

[0554] Therefore, at this stage, the enlarging area and the displaying area assigned to the master device 1 are also 
those determined on the basis of the enlargement factor N equal to the maximum enlargement factor is^,,. 
[0555] When video data conversion is performed in step S1 75, the parameter z stored in the parameter memory 168 
20 of the signal processor 137 (Fig. 22) is that determined in step S 170 so as to correspond to the maximum enlargement 
factor N^x. 

[0556] Thus, in step S1 76, the partial enlarged video data, obtained by enlarging, by a factor equal to the maximum 
enlargement factor N,^, the program video data in the enlarging area r-, determined on the basis of the enlargement 
factor N equal to the maximum enlargement factor N,^, is displayed in the displaying area R 1 determined on the basis 
25 of the enlargement factor N equal to the maximum enlargement factor N max (as for the master device 1 , the displaying 
area R 1 is identical to the display screen of the CRT 11). 

[0557] The process then proceeds to step S177. In step S177, the CPU 129 determines whether a command to 
terminate displaying the enlarged full-image video data (hereinafter, referred to as a full-image enlarging terminate 
command) has been received. 

30 [0558] If it is determined in step S1 77 that the full-image enlarging terminate command has not been received, the 
processing flow returns to step S174 to repeat the process described above. Thus, in this case, in the master device 
1 , displaying of the partial enlarged video data enlarged by a factor equal to the maximum enlargement factor N max is 
continued. 

[0559] On the other hand, if it is determined in step S1 77 that the full-image enlarging terminate command has been 
35 received, that is, if a user operates the remote commander 15 (Fig. 7) so as to display a menu screen on the CRT 11 % 
and re-click on the full image enlarge icon on the menu screen thereby causing an infrared ray corresponding to the 
full image enlarge command to be emitted from the remote commander 15 and received by the IR receiver 135 and 
finally transferred to the CPU 129, the process proceeds to step S178. In step S178, the video data conversion by the 
signal processor 137 is terminated and thus the full image enlarging process by the master device 1 is terminated. 
40 Thereafter, the program video data stored in the frame memory 127 is directly supplied to the CRT 11 via the NTSC 
encoder 128, and the program video data with a normal image size is displayed on the CRT 11. 
[0560] Referring to a flow chart shown in Fig. 41 , the full image enlarging process performed by each slave device 
2y of the scalable TV system is described below. 

[0561] In each slave device 2y (Fig. 11), first, in step S181, the CPU 149 receives, via the IEEE1394 interface 153, 
45 coefficient seed data for use in resizing transmitted in step S161 in Fig. 39 from the master device 1. Furthermore, in 
this step S181 , the CPU 149 transfers the received coefficient seed data for use in resizing to the signal processor 1 57 
(Fig. 29), which in turn stores the received coefficient seed data for use in resizing into the coefficient seed data memory 
207. In this process, the signal processor 157 saves initial coefficient seed data already existing in the coefficient seed 
data memory 207 into the available storage space of the EEPROM 157B before storing the received coefficient seed 
50 data for use resizing. 

[0562] In a case in which coefficient seed data for use in resizing already exists in the coefficient memory 207 in the 
signal processor 157 of a slave device 2|j, step S181 and also step S188 which will be described later may be skipped. 
[0563] The process then proceeds to step S182. In step S182, the CPU 149 determines whether the external input 
select command transmitted in step S164 in Fig. 39 by the master device 1 has been received. If the command has 
55 not been received, the process returns to step S182. 

[0564] However, if it is determined in step S182 that the external input select command transmitted from the master 
device 1 has been received, that is, if the external input select command transmitted from the master device 1 has 
been received by the IEEE1 394 interface 1 53 and then transferred to the CPU 149, the process proceeds to step S183. 
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In step S183, the CPU 149 makes selection of input so that program video data received by the IEEE1394 interface 
153 is supplied to the MEPG video decoder 145 via the demultiplexer 144. The process then proceeds to step S184. 
[0565] In step S184, the CPU 149 the enlarge-and-display command has been received, together with the program 
video data, the enlargement factor N, the enlarging area and the displaying area Ry, from the master device 1. 
[0566] If it is determined in step S184 that the enlarge-and-display command has been received, together with the 
program video data, the enlargement factor N, the enlarging area r y , and the displaying area R y , from the master device 
1, that is, if the enlarge-and-display command, the program video data, the enlargement factor IM, the enlarging area 
r y , and the displaying area R fj , transmitted from the master device 1 have been received by the IEEE1394 interface 
153 and then transferred to the CPU 149, the CPU 149 performs processing in accordance with the enlarge-and- 
display command such that the enlarging area r y of the program video data received together with the enlarge-and- 
display command is enlarged by a factor equal to the enlarging factor N, and the resultant partial enlarged video data 
is displayed in the displaying area Ry of the display screen of the CRT 31. 

[0567] More specifically, in step S185 after step S184, the CPU 149 stores the parameter z, having a value corre- 
sponding to the enlarging factor N received together with the enlarge-and-display command, into the parameter memory 
208 of the signal processor 157 (Fig. 29). The process then proceeds to step S186. 

[0568] In step S1 86, the CPU 1 49 controls the signal processor 1 57 (Fig. 29) so as to perform video data conversion 
on the enlarging area r^ of the program video data which has been received together with the enlarge-and-display 
command and which is stored in the frame memory 147 so that the enlarging area r y (Fig. 40B) assigned to the slave 
device 2y is enlarged. 

[0569] More specifically, in the present embodiment, if the CPU 149 receives, via the IEEE1394 interface 153, TS 
packets of program video data together with the enlarge-and-display command transmitted, in steps 169 and S174 in 
Fig. 39, from the master device 1 to the slave devices 2y, the CPU 149 supplies the TS packets to the MPEG video 
decoder 145 via the demultiplexer 144. The MPEG video decoder 145 performs MPEG-decoding on the TS packets 
to obtain program video data. The obtained program video data is stored in the frame memory 147. 
[0570] The coefficient seed data memory 207 of the signal processor 157 (Fig. 29) of the slave device 2jj has coef- 
ficient seed data for use in resizing, stored in step S181, and the signal processor 157 performs video data conversion 
on the enlarging area r y of the program video data stored in the frame memory 147, using the coefficient seed data for 
use in resizing stored in the coefficient seed data memory 207 and using the tap coefficient generated from the pa- 
rameter z stored in the parameter memory 208, so as to convert the enlarging area r y of the program video data into 
a partial enlarged video data (enlarged (resized) by the enlargement factor of N). 

[0571] In the above process, the CPU 149 controls the signal processor 157 so that the resultant partial enlarged 
video data is stored in the displaying area Ry (Fig. 40A), assigned to the slave device 1 <y of the display screen of the 
CRT 31. That is, the signal processor 157 adjusts the displaying position so that the partial enlarged video data is 
displayed in the displaying area Ry (Fig. 40A), assigned to the slave device 2y, on the display screen of the CRT 31. 
[0572] More specifically, for example, in the case of the slave device 2 1V the displaying position is adjusted so that 
the partial enlarged video data is displayed in the displaying area R^ in the bottom right corner of the display screen 
of the CRT 31 as shown in Fig. 40A. 

[0573] In this example, the video data associated with the area of the display screen of the CRT 31 of the slave 
device 2„ other than the displaying area R„ is set to the black level. In the other slave devices 2 y , the displaying 
position is adjusted in a similar manner. 

[0574] In step 5187, the signal processor 157 supplies the partial enlarged video data obtained in step S186 to the 
CRT 31 via the frame memory 147 and the NTSC encoder 148. The CRT 31 displays the received partial enlarged 
video data. 

[0575] The processing flow then returns to step S184 to repeat the process from step S184 to step S187. 
[0576] On the other hand, if it is determined in step S184 that the enlarge-and-display command, the program video 
data, the enlargement factor N, the enlarging area r^, and the displaying area Ry have not been received from the 
master device 1 , that is, if the IEEE1394 interface 153 can receive no further enlarge-and-diaplay command, program 
video data, enlargement factor N, enlarging area x iy and displaying area Ry from the master device 1, the process 
proceeds to step S188. In step S188, the signal processor 157 resets (by means of overwriting) the original coefficient 
seed data saved in the EEPROM 1 57B into the coefficient seed data memory 207 (Fig, 29). Thus the full image enlarging 
process by the slave device is ended. 

[0577] In the full image enlarging process performed by the master device shown in Fig. 39 and the slave devices 
shown in Fig. 41, rf the full image enlarging process is started when program video data is being displayed on the 
master device 1 located, for example, in the second row in the second column of the array of television sets of the 
scalable TV system as shown in Fig. 42A, the full image of the program video data displayed on the master device 1 
is gradually enlarged over the screens of the slave devices 2„ to 233 while a central part of the full image is displayed 
on the screen of the master device 1 as shown in Fig. 42 B, until an enlarged full image of the program video data is 
displayed over the whole area of the composite screen consisting of display screens of 3 X 3 television sets including 
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the master device 1 and slave devices 2^ to 2 33 as shown in Fig. 42C. 

[0578] This allows a user to view the full image of the program video data in the enlarged form and thus to view 
details of the program video data. 

[0579] However, in a practical scalable TV system, adjacent display screens of television sets are separated by 

5 frames of the cases of the respective television sets, and thus no image is displayed in the frame zones, although the 
frame zones between adjacent television sets are not shown in Fig. 42 for the purpose of simplicity, Thus, in practical 
scalable TV systems, the full image of enlarged video data is separated by such zones in which no image is displayed. 
[0580] However, human eyes have a capability of interpolate a part of an image hidden by a stripe zone with a small 
width from a displayed part of the image near the stripe zone, and thus the existence of stripe zone does not cause a 

10 serious problem in viewing the full image of program video data. 

[0581] In the full-image enlarging process, instead of obtaining an enlarged full image of video data by performing 
video data conversion using the coefficient see data for use in resizing, an enlarged full image of video data may also 
be obtained by means of simple interpolation, as in the case of the partial-image enlarging process. 
[0582] When an enlarged full image of program video data is displayed, details are represented in the displayed 

15 image only in the case in which the video data conversion is performed in steps 137 and 157 by the signal processor 
137 using the coefficient seed data for use in resizing. In a case in which program video data is enlarged by means of 
simple interpolation, details are not represented, although an enlarged full image can be displayed. That is, the image 
quality of the enlarged full image produced by means of simple interpolation is not good compared with that of the 
enlarged full image produced using the coefficient seed data for use in resizing. 

20 [0583] Although in the present embodiment the special capability is provided only when authentication described 
above with reference to Figs. 31 and 33 has been successfully passed, a limited version of special capability may be 
provided even when authentication fails. 

[0584] For example, an enlarged full image of video data produced by means of video data conversion using the 
coefficient seed data for use in resizing is provided when authentication has been passed, while an enlarged full image 

25 produced by means of simple interpolation is provided if authentication fails. 

[0585] In a case in which a scalable TV system is constructed using television sets which do not have capability of 
serving as either a master device or slave devices, although an enlarged full image produced by means of simple 
interpolation can be displayed, the image quality is not good compared with that produced by means of the video data 
conversion using the coefficient seed data for use in resizing. 

30 [0586] In contract, in the scalable TV system constructed using television sets capable of serving as a master device 
or slave devices, an enlarged full image having high quality produced by means of the video data conversion using 
the coefficient seed data for use resizing is displayed. 

[0587] This causes users, having a scalable TV system constructed using television sets incapable of operating as 
either a master device or slave devices, to have an incentive to purchase television sets capable of operating as a 

35 master device or slave devices to view an enlarged full image with high quality. 

[0588] Although in the embodiment described above, the maximum enlargement factor N^x is set such that when 
the full image of program video data initially displayed on the master device 1 is enlarged by a factor equal to the 
maximum enlargement factor N,^. the resultant enlarged image has a size equal to the size of the total screen made 
up of screens of respective television sets in the scalable TV system, the maximum enlargement factor N^x mav a,so 

40 be set by a user to an arbitrary value by operating the remote commander 15 (or the remote commander 35). 

[0589] In this case, there is a possibility that the maximum enlargement factor is set to a value which causes 
the enlarged full image of program video data to have a size greater than the size of the total screen made up of screens 
of respective television sets in the scalable TV system. Hereinafter such a value will be referred to as an oversized 
enlargement factor. If program video data is enlarged by an oversized enlargement factor, the whole of the resultant 

45 enlarged full image cannot be displayed within the total screen size of the scalable TV system. In other words, only 
some part of such an enlarged full image can be displayed. In this case, a user can specify which part of the full image 
obtained by enlargement by an oversized enlargement factor should be displayed, by operating the remote commander 
1 5 (or the remote commander 35) 

[0590] Although in the example described above, each of television sets in the scalable TV system generates video 
50 data of a partial enlarged image to be displayed on each television set, video data of all partial enlarged images to be 
displayed on the respective television sets in the scalable TV system may be generated by one television set such as 
the master device 1 or two or more particular television sets. For example, the master device 1 may generate video 
data of an enlarged full image and may transmit video data of partial enlarged images, which are particular parts of 
the video data of the enlarged full image, to the corresponding slave devices 2y via the IEEE1394 interface 133. How- 
55 ever, in this case, the master device 1 has to perform a greater amount of processing to generate video data of partial 
enlarged images to be displayed on the respective slave devices 2y in addition to the video data of a partial enlarged 
image to be displayed on the master device 1 . 

[0591] Furthermore, in the example described above, video data of television broadcast program (program video 
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data) is enlarged, video data input from an external device may also be enlarged in the full-image enlarging process 
as in the case of the partial-image enlarging process. 

[0592] Furthermore, the full image enlarging process may be performed, as with the partial image enlarging process, 
not only in such a manner that original program video data is enlarged by the same factor in both horizontal and vertical 
directions but also in such a manner that the original program video data is enlarged by different factors in horizontal 
and vertical directions, 

[0593] Although in the example described above, video data initially displayed on the master device 1 located at the 
center of the 3 X 3 arrangement of television sets of the scalable TV system is enlarged (in a total eight directions 
including an upper-leftward direction, a leftward direction, a lower-leftward direction, a downward direction, an upper- 
rig htward direction, a rightward direction, and a lower-rightward direction) toward the respective slave devices 2y sur- 
rounding the master device 1 , video data initially displayed on another television set such as the slave device 2 31 at 
the lower left location may be enlarged may be enlarged toward the slave device 2 21 at the upper location, the master 
device 1 at the upper right location, the slave device 2 32 at the right-side location and so on until the enlarged full image 
is displayed. 

[0594] In the example described above, video data of an enlarged full image (video data of partial enlarged images 
forming the enlarged full image) is generated by the master device 1 or a slave device 2^ in response to a full-image 
enlarge command issued by a user by operating the remote commander 15, the master device 1 and the respective 
slave device 2jj may always generate video data of full images enlarged by factors of N equal to 1 + ct, 1 + 2a, 1 + 
3a,..., so that full images enlarged by factors of 1 + a, 1 + 2a, 1 + 3a, ... N max are sequentially displayed in 
immediate response to an full-image enlarge command. 

[0595] The scalable TV system also has a special capability of displaying video data such that a single full image is 
displayed over all television sets of the scalable TV system. This special capability is herein referred to as on-multi- 
screen displaying. That is, this special capability is achieved by performing an on-multiscreen displaying process using 
the master device 1 and the slave devices 2. 

[0596] An on-multiscreen display command can also be issued via the menu screen, as with the partial-image enlarge 
command and the full-image enlarge command. 

[0597] More specifically, if a user operates the menu button switch 54 of the remote commander 15 (Fig. 7) (or the 
menu button switch 84 of the remote commander 35 (Fig. 8)), a menu screen is displayed on the CRT 11 of the master 
device 1 (or the CRT 31 of the slave device 2). An icon indicating an on-multiscreen display command (hereinafter, 
referred to as an on-multiscreen display icon) is displayed on the menu screen. If a user clicks on the on-multiscreen 
display icon by operating the remote commander 15, the on-multiscreen displaying process is started in the master 
device 1 and the slave devices 2. 

[0598] The on-multiscreen displaying process performed by the master device is described below with reference to 
a flow chart shown in Fig. 43. 

[0599] In the on-multiscreen display mode, as described in Fig. 42C, a single image of program video data is displayed 
over the television sets of the scalable TV system. Therefore, the on-multiscreen displaying process performed by the 
master device 1 is substantially the same as the full image enlarging process shown in Fig. 39 except that the enlarge- 
ment factor N is fixed to the maximum enlargement factor and the enlarging pitch a is ignored. 
[0600] Thus, in the on-multiscreen displaying process performed by the master device 1, steps S191 to 194 are 
performed in similar manner to steps S161 to S164 in the full image enlarging process shown in Fig. 39. 
[0601] Thereafter, the process proceeds to step S195. In step S195, the maximum enlargement factor is set 
in a similar manner as in step S165 in Fig. 39. The process then proceeds to step S196. In step S196, the CPU 129 
of the master device 1 sets the enlargement factor N to the maximum enlargement factor N,^. The process then 
proceeds to step S197. 

[0602] In step S197, the CPU 129 determines the enlarging area r 1t assigned to the master device 1, of the program 
video data, and the enlarging areas assigned to respective slave devices 2jj, of the program video data on the basis 
of the enlargement factor N set to the maximum enlargement factor N^, in a similar manner as in step S167 shown 
in Fig. 39. The process then proceeds to step S198. 

[0603] In the case of the full image enlarging process shown in Fig. 39, the enlarging area is determined in step 
S167 and the displaying area is determined in step S168. However, when the enlargement factor N is equal to the 
maximum enlargement factor N,^, the displaying area of the master device 1 is the whole area of the display 
screen of the CRT 1 1 , and the displaying area Ry of each slave device 2^ is also the whole area of the display screen 
of the CRT 31 . Thus, in this case, the displaying areas are already known and it is not required to determine them (the 
displaying areas can be regarded as having already been determined). Therefore, the on-multiscreen displaying proc- 
ess does not include the steps of determining the displaying area R 1 of the master device 1 and the displaying areas 
Rjj of the slave devices 2 x y 

[0604] In step S1 98, the CPU 1 29 sets the parameter z corresponding to the enlargement factor N set to the maximum 
enlargement factor in a similar manner to step S170 in Fig. 39 and stores it into the parameter memory 168 of 
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the signal processor 137 (Fig. 22). 

[0605] Thereafter, steps 199 to S201 are performed in a similar manner as in steps S 1 74 to S 176 in Fig. 39. As a 
result, partial enlarged video data enlarged by a factor equal to the maximum enlargement factor is displayed on 
the master device 1 . 

5 [0606] The process then proceeds to step S202. In step S202, the CPU 129 determines whether a command to 
terminate the on-multiscreen displaying process (hereinafter, referred to as a on-multiscreen display terminate com- 
mand). 

[0607] If it is determined in step S202 that the on-multiscreen display terminate command has not been received, 
the processing flow returns to step S1 99 to repeat the process described above. Thus, in this case, in the master device 
10 1 , displaying of the partial enlarged video data enlarged by a factor equal to the maximum enlargement factor t^ max is 
continued. 

[0608] On the other hand, if it is determined in step S202 that the on-multiscreen display terminate command has 
been received, that is, if a user operates the remote commander 15 (Fig. 7) so as to display a menu screen on the 
CRT 11 and re-click on the on-multiscreen display icon on the menu screen thereby causing an infrared ray corre- 

*5 sponding to the on-multiscreen display command to be emitted from the remote commander 15 and received by the 
IR receiver 135 and finally transferred to the CPU 129, the process proceeds to step S203. In step S203, the video 
data conversion process by the signal processor 137 is terminated, and thus the on-multiscreen displaying process in 
the master device 1 is terminated. Thereafter, the program video data stored in the frame memory 127 is directly 
supplied to the CRT 11 via the NTSC encoder 128, and the program video data with a normal image size is displayed 

20 on the CRT 11. 

[0609] The on-multiscreen displaying process performed by the slave devices 2y is similar to the full-image enlarging 
process performed by the slave devices 2^, described above with reference to Fig. 41, and thus a further description 
is not given herein. 

[061 0] The scalable TV system also as a special capability of performing the same process in all television sets in 
25 the scalable TV system. This special capability is achieved by performing a simultaneous control process in the master 
device 1. 

[0614] A simultaneous control command can also be issued via the menu screen, as with other commands such as 
the partial-image enlarge command. 

[0612] More specifically, rf a user operates the menu button switch 54 of the remote commander 15 (Fig. 7) (or the 
30 menu button switch 84 of the remote commander 35 (Fig. 8)), a menu screen is displayed on the CRT 1 1 of the master 
device 1 (or the CRT 31 of the slave device 2). An icon indicating the simultaneous control command (hereinafter, 
referred to as a simultaneous control icon) is displayed on the menu screen. If a user clicks on the simultaneous control 
icon by operating the remote commander 15, the master device 1 starts the simultaneous control process. 
[061 3] Referring to a flow chart shown in Fig. 44, the simultaneous control process performed by the master device 
35 1 is described below. 

[0614] In the simultaneous control process, if a command is issued by operating the remote commander 1 5 (or the 
remote commander 25), an infrared ray carrying the command is emitted from the remote commander 15. The IR 
receiver 1 5 receives the infrared ray from the remote commander 15, and the command is transferred to the CPU 129 
of the master device 1 (Fig. 10). Thus, in step S211, the CPU 129 performs a process specified by the received com- 

40 mand. The process then proceeds to step S212. 

[0615] In step S2 12, the CPU 129 determines whether the scalable TV system includes a slave device 2y capable 
of performing the process corresponding to the command which was issued by the remote commander 15 and which 
was received in step S211 (hereinafter, such a command will be referred to as a remote control command). 
[061 6] The determination in step S21 2 is performed by the CPU 129 on the basis of the capability information of the 

45 respective slave devices 2^ stored in the EEPROM 130. 

[0617] If it is determined in step S2 12 that there is a slave device 2y capable of performing the process corresponding 
to the remote control command, the process proceeds to step S213. In step S213, the CPU 129 transmits, via the 
IEEE1394 interface 133, the remote control command to all slave devices 2y capable of performing the process cor- 
responding to the remote control command. 

50 [0618] In a case in which all slave devices 2y in the scalable TV system are capable of performing the process 
corresponding to the remote control command, the remote control command is transmitted to all slave devices 2y, and 
the respective slave devices 2^ perform the process corresponding to the remote control command, that is, the same 
process as that performed by the master device 1 in step S211 . 

[061 9] On the other hand, if it is determined in step S21 2 that there is no slave devices 2y capable of performing the 
55 process corresponding to the remote control command, the process jumps to step S21 4 without performing step S21 3. 
In step S214, the CPU 129 determines whether a command for terminating a simultaneous control process (a simul- 
taneous control terminate command) has been received. 

[0620] If it is determined in step S212 that the simultaneous control terminate command has not been received, the 
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process waits for receiving a command of a particular process (remote control command) issued by the remote com- 
mander 15. If a remote control command is received, the process returns to step S211 to repeat the process described 
above. 

[0621 J On the other hand, if it is determined in step S2 12 that the simultaneous control terminate command has been 
5 received, that is, if a user operates the remote commander 15 (Fig. 7) so as to display a menu screen on the CRT 11 
and re-click on the simultaneous control icon on the menu screen thereby causing an infrared ray corresponding to 
the simultaneous control command to be emitted from the remote commander 15 and received by the I R receiver 135 
and finally transferred to the CPU 129, the simultaneous control process is terminated. 

[0622] In the simultaneous control process, when all slave devices 2^ in the scalable TV system are capable of 
10 performing a remote control command, if, for example, a command to select a particular channel is issued as a remote 
control command by operating the remote commander 15, video data of the specified channel is displayed on all 
television sets including the master device 1 and the slave devices 2 in the scalable TV system, as shown in Fig. 45A. 
If a user further operates the remote commander 15 to issue a remote control command to select another channel, 
the channel is switched in the master device 1 and all slave devices 2 in the scalable TV system, as shown in Fig. 45B. 
15 [0623] Thus, the user can control simultaneously all television sets in the scalable TV system in the same way using 
a single remote commander 1 5. 

[0624] As described earlier, the remote commander 15 may be assigned to the master device 1 and the remote 
commander 35 may be assigned to a slave device 2 iy The master device 1 can be controlled by both the remote 
controller 1 5 of the master device 1 and the remote commander 35 of the slave device 2 iy and the slave device 2^ can 
20 be controlled by both the remote controller 35 of the slave device 2jj and the remote commander 15 of the master 
device 1, 

[0625] Therefore, it is possible to control all television sets in the scalable TV system by using only a single remote 
commander 15 or 35. 

[0626] One technique of controlling all television sets using only one remote commander 15 is to set device IDs of 
25 respective television sets in the remote commander 15 and specify a television set to be controlled by inputting a 
corresponding device ID before inputting a command. However, in this technique, a user has to do a troublesome job 
to identify a television set to be controlled. 

[0627] An arbitrary television set can be controlled if the remote commander 15 assigned to the master device 1 is 
used to control the master device 1 and the remote commander 35 assigned to the slave device 2y is used to control 
30 the slave device 2 iy 

[0628] However, in this technique, as many remote commanders as 9 are needed to control the television sets in 
the scalable TV system shown in Fig. 1 A. Besides, it is not easy to recognize which remote commander can be used 
to control which television set 

[0629] The above problems can be solved if an arbitrary desired television set serving as a master device 1 or slave 
35 devices 2 (l in the scalable TV system can be controlled using an arbitrary one of remote commanders 1 5 or 35 without 
having to perform a special operation to identify a television set be controlled. 

[0630] To meet the above requirement, the scalable TV system has a special capability of automatically recognizing 
which television set a user wants to control and controlling that television set in response to a command issued by the 
remote commander 1 5 (or the remote commander 35). This special capability is realized by an individual device control 

40 process performed by the master device 1 and slave devices 2. 

[0631] A command to perform an individual device control can be issued via the menu screen. 
[0632] More specifically, rf a user operates the menu button switch 54 of the remote commander 1 5 (Fig. 7) (or the 
menu button switch 84 of the remote commander 35 (Fig. 8)), a menu screen is displayed on the CRT 11 of the master 
device 1 (or the CRT 31 of the slave device 2). An icon indicating the individual device control command (hereinafter, 

45 referred to as an individual device control icon) is displayed on the menu screen. If a user clicks on the individual device 
control icon by operating the remote commander 1 5, the individual device control process is started in the master 
device 1 and the slave devices 2. 

[0633] First, the individual device control process performed by the master device 1 is descried below with reference 
to a flow chart shown in Fig. 46. 

50 [0634] In the individual device control process performed by the master device 1 (Fig. 10), if the IR receiver 135 
receives an infrared ray emitted from the remote commander 1 5 (or the remote commander 35), the CPU 129 detects, 
in step S221, the intensity of the infrared ray received by the IR receiver 135. That is, if a user operates the remote 
commander 15 to control a desired one of television sets in the scalable TV system, the remote commander 15 emits 
an infrared ray corresponding to the operation performed by the user. The infrared ray is received by the I R receiver 

55 1 35 of the master device 1 and the IR receivers 1 55 of the respective slave devices 2^ (Fig. 1 1 ). In step S221 , the CPU 
129 requests the IR receiver 135 to detect the intensity of the received infrared ray. In response, the IR receiver 135 
returns data indicating the detected intensity to the CPU 129. 

[0635] The process then proceeds to step S222. In step S222, the CPU 129 requests, via the IEEE1394 interface 
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133, the respective slave devices 2y to return data indicating the detected intensity of the infrared ray emitted from the 
remote commander 15. In response to the request, the respective slave devices 2y returns data indicating the detected 
intensity of the infrared ray, and the CPU 129 acquires (receives) the data via the IEEE1394 interface 133. 
[0636] The infrared ray emitted from the remote commander 1 5 in response to the operation performed on the remote 
5 commander 1 5 by the user is received not only the master device 1 but also by the respective slave devices 2y, as 
described above, and thus, in the present step S222, the CPU 129 acquires the infrared ray intensity detected by each 
slave devices 2 <y 

[0637] The process then proceeds to step S223. In step S223, the CPU 129 detects a greatest infrared ray intensity 
among values of intensity including the intensity, detected in step S221, of the infrared ray received by the master 
10 device 1, the intensities, acquired in step S222, of the infrared ray received by the respective slave devices 2^. The 
process then proceeds to step S224. 

[0638] In step S224, the CPU 129 determines whether the greatest intensity is detected by the master device 1 or 
a slave device 2 (hereinafter, a television set which has received an infrared ray with a greatest intensity will be referred 
to as a greatest-intensity device). 
15 [0639] In the case in which it is determined in step S224 that the greatest-intensity device is the master device 1, 
the process proceeds to step S225. In step S225, the CPU 129 determines that the command indicated by the infrared 
ray received by the IR receiver 135 has been issued to the master device 1, and the CPU 129 performs a process 
corresponding to the command. 

[0640] On the other hand, if it is determined in step S224 that the greatest-intensity device is a slave device 2, the 
20 process proceeds to step S226. In step S226, the CPU 129 determines that the command indicated by the infrared 
ray received by the IR receiver 135 was issued in order to control the slave device 2 V) which detected the greatest 
intensity, and the CPU 129 transfers the command to the slave device 2y which detected the greatest intensity, via the 
IEEE1394 interface 133. 

[0641] In response, the slave device 2y which detected the greatest intensity performs a process corresponding to 
25 thevcommand indicated by the infrared ray emitted from the remote commander 15, as will be described later with 
reference to Fig. 47. 

[0642] When a user wants to control a particular one of television sets in the scalable TV system by operating the 
remote commander 15 (or the remote commander 35), the user generally points the remote commander 15 toward 
the television set to be controlled. 
30 [0643] In this case, if the directivity of the infrared ray emitted from the remote commander 15 (or the remote com- 
mander 35) is sharp enough, a television set aimed at by the principal axis of the infrared ray emitted from the remote 
commander 15 that is. a television set which detects the greatest infrared intensity, is a television set the user wants 
to control. 

[0644] Therefore, if the greatest-intensity device performs a process in accordance with the command indicated by 
35 the infrared ray emitted from the remote commander 15, the operation intended by the user is performed by the tele- 
vision set intended by the user is correctly performed. 

[0645] More specifically, for example, if a user issues a channel selection command or a sound volume control 
command by operating the remote commander 15 aimed at the master device 1, the master device 1 detects the 
greatest infrared ray intensity, and thus the channel selection control or sound volume control is performed for the 

40 master device 1 . On the other hand, if a user issues a channel selection command or a sound volume control command 
by operating the remote commander 15 aimed at a slave device 2g, the slave device 2 ti detects the greatest infrared 
ray intensity, and thus the channel selection control or sound volume control is performed for the slave device 3j, 
[0646] After completion of step S225 or S226, the process proceeds to step S227. In step S227, the CPU 129 de- 
termines whether a command to terminate the individual device control process (hereinafter, referred to as an individual 

45 device control terminate command). 

[0647] On the other hand, if it is determined in step S227 that the individual device control terminate command has 
not been received, the process returns to step S221 after an infrared ray emitted from the remote commander 15 is 
next received by the I R receiver 135, to repeat the process described above. 

[0648] On the other hand, if it is determined in step S227 that the individual device control terminate command has 
50 been received, that is, if a user operates the remote commander 15 (Fig. 7) so as to display a menu screen on the 
CRT 11 and re-click on the individual device control icon on the menu screen thereby causing an infrared ray corre- 
sponding to the individual device control command to be emitted from the remote commander 15 and received by the 
IR receiver 135 and finally transferred to the CPU 129, the process proceeds to step S228. In step S228, the CPU 129 
transmits the individual device control terminate command to the respective slave devices 2y via the I EEE1 394 interface 
55 133 and terminates the individual device control process in the master device 1. 

[0649] Now referring to a flow chart shown in Fig. 47, the individual device control process performed by a slave 
device is described. 

[0650] In the individual device control process performed by the slave device 2 (Fig. 11), if the I R receiver 155 receives 
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an infrared ray emitted from the remote commander 15 (or the remote commander 35), the CPU 149 detects, in step 
S231 , the intensity of the infrared ray received by the I R receiver 155. That is, if a user operates the remote commander 
15 to control a desired one of television sets in the scalable TV system, the remote commander 15 emits an infrared 
ray corresponding to the operation performed by the user. The infrared ray is received by the IR receiver 155 of the 
slave device 2, as described above. In step S231 described above, the CPU 149 requests the IR receiver 155 to detect 
the intensity of the received infrared ray. The detected intensity of the infrared ray is returned to the CPU 149. 
[0651 J The process then proceeds to step S232. In step S232, in response to receiving a request for the intensity of 
the infrared ray from the master device 1, the CPU 149 transmits data indicating the infrared ray intensity detected in 
step S231, via the IEEE1394 interface 153. The intensity of the infrared ray transmitted in this step S232 is acquired 
(received) in step S222 in the above-described process (Fig. 46) performed by the master device 1. 
[0652] The process then proceeds to step S233, In step S233, the CPU 149 determines whether a command has 
been received from the master device 1. As described earlier, in step S226 or S228 shown in Fig. 46, a command is 
transmitted from the master device 1 to the slave device 2, and thus, in the present step S233, the CPU 1 49 determines 
whether such a command from the master device 1 has been received. 

[0653] If it is determined in step S233 that a command has not been received from the master device 1 , the process 
returns to step S233. 

[0654] On the other hand, if it is determined in step S233 that a command has been received from the master device 
1, that is, if the command transmitted from the master device 1 has been received by the IEEE1394 interface 153 and 
transferred to the CPU 149, the process proceeds to step S234. In step S234, the CPU 149 determined whether the 
received command is the individual device control terminate command. 

[0655] If it is determined in step S234 that the command received from the master device 1 is not the individual 
device control terminate command, the process proceeds to step S235. In step S235, the CPU 149 performs a process 
corresponding to the command received from the master device 1 . Thereafter, the process returns to step S233. 
[0656] Thus, as described above with reference to Fig. 46, if a user operates the remote commander 1 5 aimed to a 
particular slave device 2, that slave device 2 performs a process (such as a channel selection or a volume control) 
corresponding to the operation of the remote commander 1 5 performed by the user. 

[0657] On the other hand, if it is determined in step S234 that the command received from the master device 1 is 
the individual device control terminate command, the slave device 2 terminates the individual device control process. 
[0658] If the directivity of the infrared ray emitted from the remote commander 15 (or the remote commander 35) 
used herein is sharp enough, the scalable TV system can detect which one of television sets receives the highest 
intensity of an infrared ray emitted from the remote commander 15 thereby determining (detecting) which one of tele- 
vision sets a user wants to control. This makes it possible for the user to control any desirable one of television sets 
serving as the master device 1 or slave devices 2y in the scalable TV system using the remote commander 15 of the 
master device 1 or an arbitrary one of remote commanders 35 of the slave devices 2 Vj without necessitating the user 
to perform an additional operation to specify a television set the user wants to control. 

[0659] The individual device control process makes it possible for a plurality of users to view different programs. For 
example, a user may view a certain desired program PGMA on a slave device 2jj by selecting a channel using a remote 
commander 15, while another user may view another program PGMB on another slave device 2^ by selecting a 
channel using a remote commander 35. 

[0660] In this case, video data of different programs are displayed on the CRTs 31 of the slave devices 2y and 2 W 
(Fig. 11). Even in a case in which the slave devices 2 Vi and 2 M are located adjacent to each other, displaying different 
video data on the slave devices 2^ and 2 W does not cause a significant problem. 

[0661] When video data of a program PGMA is displayed on the slave device 2jj and video data of a program PGMB 
is displayed on the slave device 2^, both images are within the fields of vision of users A and B. 
[0662] However, when the user A is viewing the video data of the program PGMA displayed on the slave device 2^, 
the video data of the program PGMB displayed on the slave device 2^ is masked. Similarly, for the user B viewing the 
video data of the program PGMB displayed on the slave device 2^, the video data of the program PGMA displayed 
on the slave device 2q is masked. 

[0663] Thus, for the user A viewing the video data of the program PGMA displayed on the slave device 2^, the video 
data of the program PGMB displayed on the different slave device 2^ does not result in significant disturbance, Sim- 
ilarly, for the user B viewing the video data of the program PGMB displayed on the slave device 2^, the video data of 
the program PGMA displayed on the different slave device 2y does not result in significant disturbance. 
[0664] However, the problem is in that different audio data associated with difference video data are output. That is, 
audio data of the program PGMA is output from speaker units 32L and 32R of the slave device 2^, while different audio 
data of the program PGMB is output from speaker units 32L and 32R of the slave device 2^. 

[0665] Although human ears have the capability of listening only to a particular sound/voice when different sounds/ 
voices are generated simultaneously, as known as the cocktail party effect, a sound/voice other than a desired sound/ 
voice serves as noise which results in disturbance in listening to the desired sound/voice. In a case in which the power 
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of a desired sound/voice is too low, it is masked by another sound/voice having higher power, and a user cannot hear 
the desired sound/voice. 

[0666] To avoid the above problem, the scalable TV system has a special capability. That is, when different users 
are watching different television sets, such as a master device 1 and a slave device 2, the principal axis of directivity 
5 of speaker units 12L and 1 2R of the master device 1 (speakers thereof) is directed to a user watching the master device 

1 so that the user can easily listen to a sound/voice output from the speaker units 12L and 12R, while the principal 
axis of directivity of speaker units 32L and 32R of the slave device 2 is directed to a user watching the slave device 2. 
[0667] The speaker units 12L and 12R of the master device 1 (Fig. 10) are designed to have very sharp directivity, 
and the principal axis of directivity can be changed to a desired direction by mechanically changing the orientation of 

10 the speaker units 12L and 12R using the unit driver 138. Similarly, the speaker units 32L and 32R of the slave device 

2 are also designed to have very sharp directivity, and the principal axis of directivity can be changed to a desired 
direction by mechanically changing the orientation of the speaker units 32L and 32R using the unit driver 158. 
[0668] The speaker control can be performed only when the scalable TV system is operating in the individual device 
control mode described earlier with reference to Figs. 46 and 47, That is, the speaker control process and the individual 

15 device control process are performed in parallel. 

[0669] The speaker control process performed by the master device is described below with reference to a flow chart 
shown in Fig. 48. 

[0670] In the speaker control process performed by the master device, rf the IR receiver 135 receives an infrared 
ray emitted from the remote commander 15 (or the remote commander 35), the CPU 129 detects, in step S241, the 

20 intensity of the infrared ray received by the I R receiver 135. That is, if a user operates the remote commander 15 to 
control a desired one of television sets in the scalable TV system, the remote commander 15 emits an infrared ray 
corresponding to the operation performed by the user. The infrared ray is received by the IR receiver 135 of the master 
device 1 and the IR receivers 155 of the respective slave devices 2 tj (Fig. 11). In step S241, the CPU 129 requests the 
IR receiver 135 to detect the intensity of the received infrared ray. The detected intensity of the infrared ray is returned 

25 to the CPU 129. 

[0671] The process then proceeds to step S242. In step S242, the CPU 129 requests, via the IEEE 1394 interface 
133, the respective slave devices 2y to return data indicating the detected intensity of the infrared ray emitted from the 
remote commander 1 5. In response to the request, the respective slave devices 2y returns data indicating the detected 
intensity of the infrared ray, and the CPU 129 acquires (receives) the data via the IEEE1394 interface 133. 
30 [0672] The infrared ray emitted from the remote commander 1 5 in response to the operation performed on the remote 
commander 15 by the user is received not only the master device 1 but also by the respective slave devices 2^, as 
described above, and thus, in the present step S242, the CPU 129 acquires the infrared ray intensity detected by each 
slave devices 2 iy 

[0673] Steps S241 and S242 in the speaker control process of the master device 1 are performed in a similar manner 
35 to steps 5221 and S222, respectively, in the individual control process performed by the master device 1 shown in Fig. 
46. Therefore, in the speaker control process of the master device 1, steps S241 and S242 may not be performed, 
and the infrared ray intensity detected in steps S221 and S222 in the individual device control process of the master 
device 1 may be employed. 

[0674] The process then proceeds to step S243. In step S243, the CPU 129 select arbitrary three intensities from 
40 those including the intensity of the infrared ray detected in step S241 by the master device 1 and intensities detected 
in step S242 by the respective slave devices 2^ For example, the CPU 129 selects first to third greatest intensities. 
The process then proceeds to step S244. 

[0675] In step S244, the CPU 129 calculates the distance corresponding to each of the three intensities (first to third 
greatest intensities) selected step S243. The process then proceeds to step S245. 
45 [0676] When the infrared ray emitted from the remote commander 15 is received by a certain television set, the 
intensity of the received infrared ray depends on the distance between the remote commander 15 and that television 
set (more precisely, the IR receiver 135 or 155 of the television set). 

[0677] The EEPROM 150 of the master device 1 (Fig. 10) stores an intensity-distance table, such as that shown in 
Fig. 49, indicating the correspondence between the intensity of the infrared ray transmitted from the remote oommander 
so 15 and received by a television set and the distance between the remote commander 15 and the television set. In step 
5244, the CPU 129 determines the distance corresponding to each of three intensities (first to third greatest intensities) 
by referring to the intensity-distance table. 

[0678] The intensity-distance table can be produced by measuring the intensity of the infrared ray emitted from the 
remote commander 1 5 and received by a television set for various distances between the remote commander 1 5 and 
55 the television set. 

[0679] Referring again to Fig. 48, in step S245, the CPU 129 determines the location of the remote commander 15 

on the basis of the distances corresponding to the first to third greatest intensities of the infrared ray. 

[0680] Referring to Fig. 50, a method of determining the location of the remote commander 15 on the basis of the 
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distances corresponding to the first to third greatest intensities of the infrared ray is described below. In the following 
description, for simplicity, it is assumed that the location is determined on the basis of the first and second greatest 
intensities. 

[0681] Herein, let us assume that the master device 1 has detected the greatest intensity and a slave device 
5 located on the right side (as viewed from the front side of the scalable TV system) of the master device 1 has detected 

the next greatest intensity. Let r, denote the distance corresponding to the intensity detected by the master device 1 

and let r^j denote the distance corresponding to the intensity detected by the slave device 233. 

[0682] In a two-dimensional plane, as shown in Fig. 50, the remote commander 15 must locate on the perimeter of 

a circle c 1 whose radius is equal to r, and whose center is located at point P, at which the IR receiver 135 of the master 
10 device 1 detects the infrared ray, and the remote commander 15 must locate on the perimeter of a circle whose 

radius is equal to r^j and whose center is located at a point P23 at which the IR receiver 155 of the slave device 223 

detects the infrared ray. 

[0683] As a result, the remote commander 15 must locate at the intersection P u of the perimeters of circles q and 
C23. Thus, the location of the remote commander 1 5 can be given by point P v . 

15 [0684] In the present example, the location of the remote commander 1 5 in the two-dimensional plane is determined 
from the two values of intensity. Similarly, the location, in a three-dimensional space, of the remote commander 1 5 can 
be determined from the intersection of the surfaces of three Spheres with radii corresponding to three values of intensity. 
[0685] Referring again to Fig. 48, after completion of detecting the location of the remote commander 1 5 in step 
S245, the process proceeds to step S246. In step S246, the CPU 129 detects the greatest intensity from the values 

20 of intensity including the intensity detected by the master device 1 in step S241 and intensities of the infrared ray 
detected by slave devices 2y and acquired in step S242. The detection of the greatest intensity of the infrared ray in 
step S246 may not be performed, and the result of detection of the greatest intensity in step S223 in Fig. 46 may be 
employed. 

[0686] In step S246, the CPU 1 29 further determines whether the greatest intensity is detected by the master device 
25 1 or a slave device 2 (that is, the CPU 129 detects a television set which has detected the greatest intensity). 

[0687] If it is determined in step S246 that the greatest intensity was detected by the master device 1, the process 
proceeds to step S247. In step S247, the CPU 129 controls the unit driver 138 so as to adjust the positions of the 
speaker units 12L and 12R of the master device 1 so that the principal axis of the directivity is directed to the location 
of the remote commander 15 (the location of the user) detected in step S245. Thereafter, the process returns to step 
30 S241. 

[0688] In step S247, described above, the unit driver 138 rotates, under the control of the CPU 129, the speaker 
units 12L and 12R in a panning or tilting direction so that the principal axis of directivity is directed to the location of 
the user. 

[0689] On the other hand, if it is determined in step S246 that the greatest intensity was detected by a slave device 
35 2, the process proceeds to step S248. In step S248, the CPU 129 transmits, via the IEEE 1 394 interface 1 33, a speaker 
control command to the slave device 2y to adjust the direction of the directivity of the speaker units 32L and 32 R so 
that the principal axis of directivity is directed toward the location of a user. The process then returns to step S241. 
[0690] In response, in this specific case, the slave device 2^, which detected the greatest infrared ray intensity, rotates 
the speaker units 32L and 32R in the panning direction or in the tilting direction so that the principal axis of directivity 
40 of the speaker units 32 L and 32 R is directed toward the location of the user, as will be described later with reference 
to Fig. 51. 

[0691 ] As described above, when a user wants to control a particular one of television sets in the scalable TV system 
by operating the remote commander 15 (or the remote commander 35), the user generally points the remote com- 
mander 1 5 toward the television set to be controlled. 
45 [0692] In this case, if the directivity of the infrared ray emitted from the remote commander 15 (or the remote com- 
mander 35) is sharp enough, a television set aimed at by the principal axis of the infrared ray emitted from the remote 
commander 15 that is, a television set which detects the greatest infrared intensity, is a television set the user wants 
to control. 

[0693] That is, a television set which detects the greatest infrared ray intensity can be regarded as a television set 
50 outputting video data and audio data of a program being watched and listened to by the user who operated the remote 
commander 15. Thus, the orientations of the speaker units 12L and 12R of the master device 1 or the speaker units 
32L and 32R of a slave device 2, determined to be the greatest-intensity device, are adjusted so that the principal axis 
of directivity of speaker units is directed toward the user who operated the remote commander 15 thereby making it 
possible for the user to listen to the audio data more clearly. 
55 [0694] The speaker control process performed by a slave device 2 is described below with reference to a flow chart 
shown in Fig. 51. 

[0695] In the speaker control process performed the slave device 2 (Fig. 11), if the IR receiver 155 receives an 
infrared ray emitted from the remote commander 15 (or the remote commander 35), the CPU 149 detects, in step 
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S251 , the intensity or the infrared ray received by the IR receiver 1 55. That is, if a user operates the remote commander 
15 to control a desired one of television sets in the scalable TV system, the remote commander 15 emits an infrared 
ray corresponding to the operation performed by the user. The infrared ray is received by the IR receiver 155 of the 
slave device 2, as described above. In step S251 , the CPU 129 requests the IR receiver 155 to detect the intensity of 

5 the received infrared ray. In response, the IR receiver 1 55 returns data indicating the detected intensity to the CPU 129. 
[0696] The process then proceeds to step S252. In step S252, in response to receiving a request for the intensity of 
the infrared ray from the master device 1, the CPU 149 transmits data indicating the infrared ray intensity detected in 
step S251, via the IEEE1394 interface 153. The data indicating the infrared ray intensity transmitted in this step S252 
is acquired (received) in step S242 in Fig. 48, described earlier 

10 [0697] Steps S251 and S252 in the speaker control process of the slave device 2 are performed in a similar manner 
to steps S231 and S232, respectively, in the individual control process performed by the slave device 2 shown in Fig. 
47. Therefore, in the speaker control process of the slave device 2, steps S251 and S252 may not be performed, and 
the infrared ray intensity detected in steps S231 and S232 in the individual device control process of the slave device 
2 may be employed. 

15 [0698] The process then proceeds to step S253. In step S253, the CPU 149 determines whether the speaker control 
command has been received from the master device 1 . As described earlier, in step S248 shown in Fig. 48, the speaker 
control command is transmitted from the master device 1 to the slave device 2, and, in the present step S253, the CPU 
149 determines whether the speaker control command transmitted from the master device 1 has been received. 
[0699] If it is determined in step S253 that the speaker control command from the master device 1 has not been 

20 received, the process returns to step S251. 

[0700] On the other hand, if it is determined in step S253 that a speaker control command has been received from 
the master device 1, that is, if the speaker control command transmitted from the master device 1 has been received 
by the IEEE1394 interface 153 and transferred to the CPU 149, the process proceeds to step S254. In step S254, in 
accordance with the speaker control command, the CPU 149 controls the unit driver 1 58 so as to adjust the positions 

25 of the speaker units 32L and 32R of the slave device 2 so that the principal axis of the directivity is directed to the 
location of the remote commander 15 (the location of the user) detected in step S245 shown in Fig. 48. Thereafter, the 
process returns to step S251 . 

[0701] In step S254, described above, the unit driver 158 rotates, under the control of the CPU 149, the speaker 
units 32L and 32R in a panning or tilting direction so that the principal axis of directivity is directed to the location of 
30 the user. 

[0702] Thus, in this specific case, the slave device 2 adjusts the directivity of the speaker units 32L and 32R so that 
the principal axis of the directivity is directed to the location of the user who has operated the remote commander 15, 
that is, who is listening to audio data associated and viewing video data of a program being output by the slave device 
2 thereby making it possible for the user to listen to the audio data more clearly. 
35 [0703] The speaker control process shown in Fig. 48 or 51 is ended when the individual device control process shown 
in Fig. 46 or 47 is ended. 

[0704] Although in the embodiment described above, only the direction of the principal axis of the directivity of the 
speaker units 12L and 12R (or the speaker units 32L and 32R) is controlled depending on the location of a user, the 
sound volume of the speaker units 1 2L and 12R may also be controlled. For example, the volume of the sound output 
40 from the speaker units 1 2L and 12R may be increased with the distance between the user and the television set being 
viewed by the user. 

[0705] Although in the embodiment described above, the location of the remote commander 15 (the location of a 
user) is determined on the basis of the intensities, detected by the television sets, of an infrared ray emitted from the 
remote commander 15, the location of the remote commander 15 may be detected by another method. An example 
45 is to use a GPS (Global Positioning System) and another example is to emit an ultrasonic wave from the respective 
television sets and detect an ultrasonic wave returned from the remote commander 15. 

[0706] Although in the speaker control process according to the embodiment described above, speaker units 12L 
and 12R (and speaker units 32L and 32R) having sharp directivity are used, and the speaker units 12L and 12R are 
rotated in the panning direction or the tilting direction using the unit driver 1 38 (or the unit driver 1 58) so that the principal 
50 axis of the directivity is directed to a desirable direction (to the location of the user), the principal axis of directivity may 
also be controlled electronically. 

[0707] Fig. 52 shows an example of a manner of electrically controlling the principal axis of directivity of a speaker 
unit 12L. The directivity of other speaker units 12R, 32L and 32R can also be controlled in a similar manner to the 
speaker unit 12L, and thus in the following description, only the control of the speaker unit 12L is discussed. 
55 [0708] In the example shown in Fig. 52, audio data output from the MPEG audio decoder 126 (Fig. 10) is supplied 
to digital filters 211, and 211 2 . The tap coefficients of the digital filters 21^ and 211 2 are set by the unit driver 138 (Fig. 
10), and the digital filters 21^ and 211 2 filter the same audio data applied to the digital filters 211 n and 211 2 by using 
the tap coefficients set by the unit driver 1 38 so as to delay the audio data by particular delay times for each frequency 
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component of the audio data. The resultant delayed audio data output from the respective digital filters 21^ and 211 2 
are supplied to the speakers 212-| and 212 2 . 

[0709] The speakers 21 2^ and 212 2 are both of the nondirectional type, and they emit sounds in accordance with 
the audio data output from the digital filters 211 and 21 1 2 , respectively. 

5 [0710] Herein, let Y1 and Y2 be the principal axes of the two speakers 212 n and 212 2 , respectively, of the speaker 
unit 12L. The speakers and 212 2 are placed so that the principal axes Y1 and Y2 extend in parallel In a two- 
dimensional plane (the page of the drawing, in this specific example) and so that cones (vibrating plates) of the re- 
spective speakers 212! and 212 2 are located in the same plane perpendicular to the axes Y1 and Y2. 
[071 1 J Herein, let a denote the distance between the principal axes Y1 and Y2 (axis-to-axis distance) and let 0 denote 

10 the angle (radiation angle) as measured in a counterclockwise direction in a two-dimensional plane with respect to the 
principal axis Y1 or Y2. 

[0712] If audio data including only a single frequency component such as a sinusoidal signal, is applied to the speaker 
unit 12L, the sinusoidal signal is filtered by the digital filters 21^ and 21 1 2 thereby producing delays D1 and D2. The 
resultant sinusoidal signals delayed by D1 and D2 are applied to the speakers 212, and 212 2 . 
1 5 [071 3J In this case, sound waves output from the respective speakers 2 1 2, and 21 2 2 interfere with each other If D2 
> D1, there is a time difference (delay time difference) equal to D2 - D1 between the sound waves output from the 
respective speakers 212 n and 212 2 . On the other hand, sound waves propagating in the directions Y11 and Y12 with 
an angle 0 with respect to the principal axes Y1 and Y2 of the respective speakers 21 2-, and 2 1 2 2 experience a difference 
in propagation path length. 

20 [0714] As a result, the phase difference between the two sound waves varies depending on the location (listening 
point) where a user receives the two sound waves originating from the speakers 21 2 y and 212 2 . The phase difference 
between the two sound waves can become zero at a certain listening point. In this case, the effective amplitude of the 
sound wave becomes twice that of the sound wave which would be obtained if the sound wave were output from a 
signal speaker (either the speaker 21 2, or 212 2 ). However, at a different listening point, the phase difference between 

25 the two sound waves can become 180°. In this case, the resultant amplitude becomes zero, and thus no sound is 
heard. This means that the total volume of sound generated by the speakers 212., and 21 2 2 has directivity. 
[0715] Figs. 53 and 53 show examples of directivity of the total volume of sound generated by the speakers 212 t 
and 21 2 2 . In Figs. 53 and 54, the sound volume is normalized with respect to the maximum sound volume (0 dB). 
[071 6] Fig. 53 shows the directivity obtained when the axis-to-axis distance is set to 1 0 cm, the delay time difference 

30 D2 - D1 is set to a/C, and a sinusoidal signal with a frequency of 1000 Hz is applied, where C denotes the acoustic 
velocity (assumed to be equal to 340 m/s). 

[0717] In the example shown in Fig. 53, the maximum volume is obtained in a range in which the angle 6 is greater 
than 30°. At a location at which the angle 0 becomes equal to -45°, the sound volume becomes substantially zero (null). 
[0718] Fig. 54 shows the sound directivity obtained for similar conditions to those employed in Fig. 53, except that 

35 a sinusoidal signal with a frequency of 5000 Hz is applied. 

[0719] In the example shown in Fig. 54, a main beam appears in a range in which the angle 9 is greater than 45°, 
and a sub beam (grating beam) with a magnitude similar to that of the main beam appears in a range in which the 
angle 6 is 0 to 45°. The reason why such a sub beam appears is that the phase difference between two sound waves 
becomes equal to an integral multiple of the wavelength of the sinusoidal wave with a frequency of 5000 Hz, and thus 

to two sound waves are added together in phase. 

[0720] In general, when the distance of the listening point from the speaker 21 2 A and that from the speaker 21 2 2 are 
much greater than the axis-to-axis distance a, if the following equation holds, two sound waves originating from the 
speakers 212! and 21 2 2 are added together in phase and a sub beam with a magnitude equal to that of the main beam 
appears. 

45 

a/C X (1 - cosO) = 1/f x n (26) 

where f is the frequency of an input signal, and n is an integer equal to or greater than 0. 
50 [0721] In equation (26), a main beam occurs when n = 0. 

[0722] For example, if the frequency f is 1000 Hz, equation (26) is satisfied only when n = 0. Therefore, in this case, 
only a main beam appears but no sub beam appears. 

[0723] When n = 1, the frequency which satisfies equation (26) is given by f = C/(a(1 - cosO)). That is, when the 
frequency is equal to this value, a sub beam appears. In the example shown in Fig. 53, this frequency is equal to about 
55 1700 Hz, at which the axis-to-axis distance a becomes equal to one-half the wavelength of the sound wave. 

[0724] In the speaker unit 12L shown in Fig. 52, as described above, input audio data is delayed by the digital filters 
21^ and 21 1 2 for respective frequency components, and audio data having delay time difference D2 - D1 for respective 
frequency components is output from the speakers 212! and 21 2 2 so that the total sound volume characteristic of the 
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speakers 212! and 21 2 2 has directivity. The direction of the main beam and the null direction for each frequency 
component can be changed by controlling the delay time difference at that frequency. 

[0725] That is, the direction of the principal axis of directivity of the speaker unit 12L can be changed by changing 

the tap coefficients applied to the digital filters 21 1 n and 21 1 2 . 
5 [0726] Therefore, by applying proper tap coefficients from the unit driver 138 to the digital filters 21 1 ^ and 21 1 2 , it is 

possible to direct the principal axis of directivity of the speaker unit 12L into a desired direction. 

[0727] Although in the example described above, the principal axis of directivity is controlled by means of using 

interference between two sound waves emitted from the two speakers 212! and 21 2 2 disposed in the speaker unit 

12L, each of speaker units 12L and 12R may be formed so as to include a single speaker, and the direction of the 
10 principal axis of directivity may be controlled by means of using interference between two sound waves emitted from 

the speaker of the speaker unit 12L and the speaker of the speaker unit 12R. 

[0728] The speaker unit 12L may be formed as to include an array of three or more speakers so that the speaker 
unit 12L has sharper directivity. 

[0729] In the embodiment described above, the location of the remote controller 1 5 (the location of a user) is deter- 
15 mined on the basis of intensity, detected by the master device 1 and slave devices 2, of the infrared ray emitted from 
the remote controller 15, and the principal axis of directivity of the speaker units 12L and 12R or the speaker units 32L 
and 32R is directed toward the location of the remote commander 15. However, to adjust the positions of the set of 
speaker units 12L and 12R or the set of speaker units 32L and 32R so that the principal axis of directivity is directed 
toward the remote commander 15, it is not necessarily needed to detect the location of the remote commander 15 but 
20 jt is needed only to detect the direction from the master device 1 or the slave device 2 to the remote commander 1 5. 
[0730] Referring to Figs. 55 and 56, a method of detecting the direction of the remote commander 1 5 as viewed from 
the master device 1 (or a slave device 2) is described below. 

[0731] As shown in Fig. 55, the direction of the remote commander 15 as viewed from the master device 1 can be 
detected on the basis of the infrared ray detected by two infrared detectors 1 35A and 1 35B, disposed on the I R receiver 
25 135 of the master device 1 (Fig. 10) such that the infrared detectors 135 are spaced from each other by a particular 
distance D. 

[0732].. When the distance from the master device 1 to the remote commander 1 5 is much greater than the distance 
D between tthe infrared detectors 135A and 135B, an infrared ray IRa emitted from the remote commander 15 and 
incident on the infrared detector 135A and an infrared ray IRb emitted from the remote commander 15 and incident on 

30 the infrared detector 135B can be regarded as being parallel to each other. 

[0733] Herein, as shown in Fig. 55, let $ be the angle of the infrared ray IRb emitted from the remote commander 
15 and incident on the infrared detectors 135A and 135B with respect to a line passing though the infrared detectors 
135A and 135B. The difference d in the propagation path between the infrared ray IRa emitted from the remote com- 
mander 15 and incident on the infrared detector 135A and the infrared ray IRb emitted from the remote commander 

35 15 and incident on the infrared detector 135B is given by Dcos4>. 

[0734] If the velocity of light is denoted by c, and the difference between times at which the Infrared detectors 135A 
and 135B receive the infrared rays IRa and IRb emitted from the remote commander 15 is denoted by x, the difference 
d in propagation path is given by ex. 

[0735] Therefore, the angle 4>, that is the direction of the remote commander 15 is given by arccos (xc/D). That is, 
40 the direction <J> of the remote commander 1 5 can be determined by measuring the difference x in times at which the 

infrared detectors 135A and 135B receive the infrared rays IRa and IRb emitted from the remote commander 15. 

[0736] The direction of the remote commander 1 5 as viewed from the master device 1 (or a slave device 2) can also 

be determined by constructing the IR receiver 135 (or the IR receiver 155) in such a manner as shown in Fig. 56. 

[0737] That is, in the example shown in Fig. 56, the IR receiver 1 35 is made up of an infrared line sensor 221 having 
45 a plurality of pixels serving as infrared detectors and a lens 222 for focusing the infrared ray IRc onto the infrared line 

sensor 221. 

[0738] The infrared line sensor 221 is placed on the optical axis of the lens 222. 

[0739] In the IR receiver 1 35 constructed in the above-described manner, an infrared ray I Rc emitted from the remote 
commander is incident on the infrared line sensor 221 via the lens 222 and detected by a pixel at a particular location 
50 on the infrared line sensor 

[0740] Which one of pixels on the infrared line sensor 221 detects the infrared ray IRc depends on the incidence 
angle a of the infrared ray incident on the infrared line sensor 221 . That is, the detection position varies depending on 
the incidence angle a. 

[0741] If the distance between the detection position and the intersection point of the infrared line sensor 221 and 
55 the optical axis of the lens 222 is denoted by r, and the distance between the infrared line sensor 221 and the lens 222 
is denoted by S, the incidence angle a, that is, the angle of the remote commander 15, is given by arctan(S/r). 
[0742] Therefore, the direction a of the remote commander 1 5 can be determined by measuring the distance r be- 
tween the pixel detecting the infrared ray IRC and the intersection of the infrared line sensor 221 and the optical axis 
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of the lens 222. 

(0743J Fig. 57 shows another example of the configuration of the master device 1 . In Fig. 57, similar parts to those 
in Fig. 10 are denoted by similar reference numerals, and similar parts are not described herein in further detail. That 
is, the master device 1 shown in Fig. 57 is similar to that shown in Fig. 10 except that the master device 1 shown in 
Fig. 57 further includes a connection detector 139. 

[0744] The connection detector 139 detects, electrically or mechanically, a connection of another television and 
informs the CPU 129 of the connection. 

[0745] In the example shown in Fig. 57, instead of detecting a connection by detecting a change in voltage of the 
IEEE1394 terminal 21^ (Fig. 3F) on the terminal panel 21, the connection with another television set is detected by the 
connection detector 139. 

[0746] Fig. 58 shows another example of the configuration of the slave device 2. In Fig. 58, similar parts to those in 
Fig. 11 are denoted by similar reference numerals, and similar parts are not described herein in further detail. That is, 
the slave device 2 shown in Fig. 58 is similar to that shown in Fig. 11 except that the slave device .2 shown in Fig. 58 
further includes a connection detector 159. 

[0747] The connection detector 159 detects, electrically or mechanically, a connection of another television and 
informs the CPU 149 of the connection. 

[0748] In the example shown in Fig. 58, as with the example shown in Fig. 57, the connection with another television 
set is detected by the connection detector 159, instead of detecting the connection by detecting a change in voltage 
of the IEEE1394 terminal 41., (Fig. 5F) on the terminal panel 41. 

[0749] The processing sequence described above may be executed by hardware or software. When the processes 
are performed by software, a software program is installed on a general-purpose computer or the like. 
[0750] Fig. 59 illustrates an embodiment of the invention in which a program used to execute the processes described 
above is installed on a computer. 

[0751] The program may be stored, in advance, on a hard disk 305 or a ROM 303 serving as a storage medium, 
which is disposed inside the computer. 

[0752] Alternatively, the program may be stored (recorded) temporarily or permanently on a removable storage me- 
dium 311 such as a floppy disk, a CD-ROM (Compact Disc Read Only Memory), an MO (Magnetooptical) disk, a DVD 
(Digital Versatile Disc), a magnetic disk, or a semiconductor memory. Such a removable storage medium 311 may be 
provided in the form of so-called package software. 

[0753] Instead of installing the program from the removable storage medium 311 onto the computer, the program 
may also be transferred to the computer from a download site via a digital broadcasting satellite by means of radio 
transmission or via a network such as an LAN (Local Area Network) or the Internet by means of wire communication. 
In this case, the computer receives, using a communication unit 308, the program transmitted in the above-described 
manner and installs the program on the hard disk 305 disposed in the computer. 

[0754] The computer includes a CPU (Central Processing Unit) 302. An input/output interface 310 is connected to 
the CPU 302 via a bus 301. If the CPU 302 receives, via the input/output interface 310, a command issued by a user 
using an input unit 307 including a keyboard, a mouse, microphone, or the like, the CPU 302 executes the a program 
stored in a ROM (Read Only Memory) 303. Alternatively, the CPU 302 may execute a program loaded in a RAM 
(Random Access Memory) 304 wherein the program may be loaded into the RAM 304 by transferring a program stored 
on the hard disk 305 into the RAM 304, or transferring a program which has been installed on the hard disk 305 after 
being received from a satellite or a network via the communication unit 308, or transferring a program which has been 
inetalled on the hard disk 305 after being read from a removable recording medium 311 loaded on a drive 309. By 
executing the program, the CPU 302 performs the process described above with reference to the flow charts or the 
block diagrams. The CPU 302 outputs, via an input/output interface 310, the result of the process, as required, to an 
output unit 306 including an LCD (Liquid Crystal Display) and/or a speaker thereby outputting the result of the process 
from the output unit 306. The result of the process may also be transmitted via the communication unit 308 or may be 
stored on the hard disk 305. 

[0755] In the present invention, the processing steps described in the program to be executed by a computer to 
perform various kinds of processing are not necessarily required to be executed in time sequence according to the 
order described in the flow chart. Instead, the processing steps may be performed in parallel or separately (by means 
of parallel processing or object processing). 

[0756] The program may be executed either by a single computer or by a plurality of computers in a distributed 
fashion. The program may be transferred to a computer at a remote location and may be executed thereby. 
[0757] The television sets constituting the scalable TV system may be of the digital type or analog type. 
[0758] The sales prices of the television seta constituting the scalable TV system may be set depending on whether 
the television set is a master device or a slave device and, in the case of slave devices, further depending on the 
number of slave devices. 

[0759] In the scalable TV system, a master device is necessary to achieve the special capabilities described earlier, 
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and thus the sales price of the master device may be set to a high value. 

[0760] After a user purchased a master device, the user is expected to purchase a certain number of slave devices 
at a first time and then an additional number of slave devices at an another time, and so on. The sales prices for a 
certain number of slave devices purchased for the first time may be set to a value which is lower than the price of the 
5 master device and higher than the price of a conventional television set. The prices for additional slave devices may 
be set to a lower value. 

[0761] A television set capable of serving as a master device in the scalable TV system may be constructed, for 
example, by adding the signal processor 137 to an ordinary digital television set and modifying the program executed 
by the CPU 129. This means that television sets for use as a master device in the scalable TV system can be easily 
10 produced on the basis of ordinary television sets, and thus a high cost merit (cost performance) can be achieved, if 
the specia ('capabilities provided by the scalable TV system are taken into account. High cost merit is also achieved 
for television sets for use as slave devices. 

[0762] The present invention can be applied not only to television sets having a built-in tuner but also displays which 
include no tuner and which are designed to display an image and output a sound/voice in according with video and 
15 audio signals supplied from the external. 

Claims 

20 1. A display apparatus, connectable with one or more other display apparatuses and having display means for dis- 
playing an image in accordance with an input video signal, comprising: 

classifying means for determining a class corresponding to a pixel of interest such that plural class reference 
pixels in the vicinity of the pixel of interest to be predicted are extracted from the input video signal and the 

25 class corresponding to the pixel of interest is determined from the extracted class reference pixels; 

prediction coefficient generation means for generating a prediction coefficient corresponding to the class de- 
termined by the classifying means; 
;j pixel prediction means for predicting the pixel of interest such that plural prediction reference pixels in the 
vicinity of the pixel of interest are extracted from the input video signal, and the pixel of interest is predicted 

30 by means of a prediction operation using the extracted plural prediction reference pixels and the prediction 

coefficient; and 

display control means for displaying an image including at least the pixel of interest on display means such 
that images displayed on the present display apparatus and said one or more other display apparatuses form, 
as a whole, a complete enlarged image of the image corresponding to the input video signal. 



2. A display apparatus according to Claim 1, wherein the display control means comprises: 



video signal generation means for generating a video signal to be used to display an enlarged image corre- 
sponding to the input video signal, using the input video signal and the predicted pixel of interest; and 
40 video signal output means for outputting at least a part of the video signal, to be used to display the enlarged 

image, to said one ore more other display apparatuses. 

3. A display apparatus according to Claim 1 , wherein the classifying means determines the class of the pixel of 
interest on the basis of the level distribution pattern of the class reference pixels. 



4. A display apparatus according to Claim 1, wherein the prediction reference pixels and the class reference pixels 
are the same. 



5. A display apparatus according to Claim 1 , further comprising authentication means for performing mutual authen- 
50 tication with said one ore more other display apparatuses; and 

at least the part of the enlarged image is displayed on said other display apparatuses only if the authentication 
has been successfully passed. 

6. A display apparatus according to Claim 1 , further comprising enlargement factor setting means for setting the 
55 enlargement factor to be employed to enlarge the input video signal to produce the enlarged video signal to be 

displayed. 

7. A display apparatus according to Claim 6, further comprising parameter setting means for setting a parameter 
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depending on the enlargement factor set by the enlargement factor setting means, 

wherein the prediction coefficient generation means generates the prediction coefficient on the basis of co- 
efficient seed data obtained in advance by means of learning. 

8. A display apparatus according to Claim 6, wherein the enlargement factor setting means sets the enlargement 
factor such that the enlargement factor gradually increases after displaying the image corresponding to the input 
video signal. 

9. A display apparatus according to Claim 6, further comprising: 

enlarging area detection means for determining an enlarging area of the image corresponding to the input 
video signal, to be enlarged by a factor equal to the enlargement factor into an enlarged video signal to be 
displayed on said one or more other display apparatuses; 

displaying area detection means for determining a displaying area of a screen of each of said one or more 
other display apparatuses, in which the corresponding enlarged image is to be displayed; and 
transmission means for transmitting data indicating the enlarging area, the displaying area, and the enlarge- 
ment factor to said one or more other display apparatuses. 

1 0. A display apparatus according to Claim 1 , further comprising; 

enlargement factor input means for inputting an enlargement factor transmitted from one of said other display 
apparatuses; and 

parameter setting means for setting a parameter in accordance with the enlargement factor received from said 
one of other display apparatuses, 

wherein the prediction coefficient generation means generates the prediction coefficient on the basis of co- 
efficient seed data obtained in advance by means of learning. 

11. A display apparatus according to Claim 10, further comprising: input means for inputting an enlarging area of an 
image corresponding to the input video signal transmitted from one of said other display apparatuses and for 
inputting a displaying area in which an enlarged image obtained by enlarging the image in the enlarging area is 
to be displayed, 

wherein the display means displays the enlarged image obtained by enlarging the image corresponding to 
the input video signal such that the enlarged image is displayed in the displaying area. 

12. A display method for a display apparatus, connectable with one or more other display apparatuses, to display an 
image in accordance with an input video signal, comprising: 

a classification step, of determining a class corresponding to a pixel of interest such that plural class reference 
pixels in the vicinity of the pixel of interest to be predicted are extracted from the input video signal and the 
class corresponding to the pixel of interest is determined from the extracted class reference pixels; 
a prediction coefficient generation step for generating a prediction coefficient corresponding to the class de- 
termined in the classification step; 

a pixel prediction step for predicting the pixel of Interest such that plural prediction reference pixels in the 
vicinity of the pixel of interest are extracted from the input video signal, and the pixel of interest is predicted 
by means of a prediction operation using the extracted plural prediction reference pixels and the prediction 
coefficient; and 

a display control step, of displaying an image including at least the pixel of interest on display means such 
that images displayed on the present display apparatus and said one or more other display apparatuses form, 
as a whole, a complete enlarged image of the image corresponding to the input video signal. 

13. A display method according to Claim 12, wherein the display control step includes: 

a video signal generation step, of generating a video signal to be used to display an enlarged image corre- 
sponding to the input video signal, using the input video signal and the predicted pixel of interest; and 
a video signal output step for outputting at least a part of the video signal, to be used to display the enlarged 
image, to said one or more other display apparatuses. 
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14. A program for causing a computer to control a display apparatus, which is connectable with one or more other 
display apparatuses, so as to display an image in accordance with an input video signal, said program comprising: 

a classification step, of determining a class corresponding to a pixel of interest such that plural class reference 
5 pixels in the vicinity of the pixel of interest to be predicted are extracted from the input video signal and the 

class corresponding to the pixel of interest is determined from the extracted class reference pixels; 
a prediction coefficient generation step for generating a prediction coefficient corresponding to the class de- 
termined in the classification step; 

a pixel prediction step for predicting the pixel of interest such that plural prediction reference pixels in the 
10 vicinity of the pixel of interest are extracted from the input video signal, and the pixel of interest is predicted 

by means of a prediction operation using the extracted plural prediction reference pixels and the prediction 
coefficient; and 

a display control step, of displaying an image including at least the pixel of intorest on display means such 
that images displayed on the present display apparatus and said one or more other display apparatuses form, 
15 as a whole, a complete enlarged image of the image corresponding to the input video signal. 

15. A storage medium including a program stored thereon for causing a computer to control a display apparatus so 
as to display an image in accordance with an input video signal input from the outside, said program comprising: 

20 a classification step, of determining a class corresponding to a pixel of interest such that plural class reference 

pixels in the vicinity of the pixel of interest to be predicted are extracted from the input video signal and the 
class corresponding to the pixel of interest is determined from the extracted class reference pixels; 
a prediction coefficient generation step for generating a prediction coefficient corresponding to the class de- 
termined in the classification step; 

25 a pixel prediction step for predicting the pixel of interest such that plural prediction reference pixels in the 

vicinity of the pixel of interest are extracted from the input video signal, and the pixel of interest is predicted 
, by means of a prediction operation using the extracted plural prediction reference pixels and the prediction 
coefficient; and 

a display control step, of displaying an image including at least the pixel of interest on display means such 
30 that images displayed on the present display apparatus and said one or more other display apparatuses form, 

as a whole, a complete enlarged image of the image corresponding to the input video signal. 

16. A display system including at least a first display apparatus and a second display apparatus connected with each 
other, 

35 the first display apparatus comprising: 

display means for displaying an image; 

classifying means for determining a class corresponding to a pixel of interest such that plural class reference 
pixels in the vicinity of the pixel of interest to be predicted are extracted from the input video signal and the 
40 class corresponding to the pixel of interest is determined from the extracted class reference pixels; 

prediction coefficient generation means for generating a prediction coefficient corresponding to the class de- 
termined by the classifying means; 

pixel prediction means for predicting the pixel of interest such that plural prediction reference pixels in the 
vicinity of the pixel of interest are extracted from the input video signal, and the pixel of interest is predicted 
45 by means of a prediction operation using the extracted plural prediction reference pixels and the prediction 

coefficient; 

display control means for displaying an image including at least the pixel of interest such that images displayed 
on the present display apparatus and the second display apparatus form, as a whole, a complete enlarged 
image of the image corresponding to the input video signal; and 
50 transmission means for transmitting at least part of the predicted pixel of interest; 

the second display apparatus comprising: 

input means for inputting at least part of the predicted pixel of interest; and 
55 display means for displaying the enlarged image including at least the pixel of interest. 

17. A display apparatus, connectable with one or more other display apparatuses and including display means for 
displaying an image, comprising: 
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input means for inputting video signal output from one of other display apparatuses; 

image enlarging means for generating, from the input video signal, an enlarged image of the image corre- 
sponding to the input video signal; 

authentication means for performing mutual authentication with said one of other display apparatuses; and 
display control means for, if the authentication has been successfully passed, displaying an enlarged image 
generated by the image enlarging means on the display means such that images displayed on the display 
apparatus and said one or more other display apparatuses form, as a whole, a complete enlarged image. 

18. A display apparatus according to Claim 17, wherein the image enlarging means generates the enlarged image 
from the input video signal by means of simple interpolation. 

19. A display apparatus according to Claim 17, wherein the image enlarging means includes: 

classifying means for determining a class corresponding to a pixel of interest such that plural class reference 
pixels in the vicinity of the pixel of interest to be predicted are extracted from the input video signal and the 
class corresponding to the pixel of interest is determined from the extracted class reference pixels; 
prediction coefficient generation means for generating a prediction coefficient corresponding to the class de- 
termined by the classifying means; and 

pixel prediction means for predicting the pixel of interest such that plural prediction reference pixels in the 
vicinity of the pixel of interest are extracted from the input video signal, and the pixel of interest is predicted 
by means of a prediction operation using the extracted plural prediction reference pixels and the prediction 
coefficient; 

and wherein the image enlarging means generates the enlarged image using the input video signal and the 
predicted pixel of interest. 

20. A display apparatus according to Claim 19, further comprising enlargement factor setting means for setting the 
enlargement factor to be employed to enlarge the input video signal to produce the enlarged video signal to be 
displayed. 

21. A display apparatus according to Claim 20, further comprising parameter setting means for setting a parameter 
depending on the enlargement factor set by the enlargement factor setting means, 

wherein the prediction coefficient generation means generates the prediction coefficient on the basis of co- 
efficient seed data obtained in advance by means of learning. 

22. A display apparatus according to Claim 20, wherein the enlargement factor setting means sets the enlargement 
factor such that the enlargement factor gradually increases after displaying the image corresponding to the input 
video signal. 

23. A display apparatus according to Claim 20, further comprising: 

enlarging area detection means for determining an enlarging area of the image corresponding to the input 
video signal, to be enlarged by a factor equal to the enlargement factor into an enlarged video signal to be 
displayed on said one or more other display apparatuses; 

displaying area detection means for determining a displaying area of a screen of each of said one or more 
other display apparatuses, in which the corresponding enlarged image is to be displayed; and 
transmission means for transmitting data indicating the enlarging area, the displaying area, and the enlarge- 
ment factor to said one or more other display apparatuses. 

24. A display apparatus according to Claim 19, further comprising transmission means for transmitting a part of the 
enlarged image to be displayed on each of said one or more other display apparatuses to corresponding each of 
said one or more other display apparatuses. 

25. A display apparatus according to Claim 20, further comprising: 

enlargement factor input means for inputting an enlargement factor transmitted from one of said other display 
apparatuses; 

parameter setting means for setting a parameter in accordance with the enlargement factor received from one 
of said other display apparatuses, 
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wherein the prediction coefficient generation means generates the prediction coefficient on the basis of co- 
efficient seed data obtained in advance by means of learning. 



26. A display apparatus according to Claim 25, further comprising input means for inputting an enlarging area of an 
5 image corresponding to the input video signal transmitted from one of said other display apparatuses and for 

inputting a displaying area in which an enlarged image obtained by enlarging the image in the enlarging area is 
to be displayed, 

wherein the display means displays the enlarged image obtained by enlarging the image corresponding to 
the input video signal such that the enlarged image is displayed in the displaying area. 

10 

27. A display method for a display apparatus, connectable with one or more other display apparatuses and including 
display means for displaying an image, to display an image, comprising: 

an input step, of inputting video signal output from one of other display apparatuses; 
15 an image enlarging step, of generating, from the input video signal, an enlarged image of the image corre- 

sponding to the input video signal; 

an authentication step, of performing mutual authentication with said one of other display apparatuses; and 
display control means for, if the authentication has been successfully passed, displaying an enlarged image 
generated by the image enlarging means on the display means such that images displayed on the display 
20 apparatus and said one or more other display apparatuses form, as a whole, a complete enlarged image. 

28. A program for causing a computer to control a display apparatus connectable with one or more display apparatuses 
and including display means for displaying an image, said program comprising: 



25 an image enlarging step, of generating, from the input video signal, an enlarged image of the image corre- 

sponding to the input video signal; 
, an authentication step, of performing mutual authentication with said one of other display apparatuses; and 
.a display control step of, if the authentication has been successfully passed, displaying an enlarged image 
generated by the image enlarging means on the display means such that images displayed on the display 
30 apparatus and said one or more other display apparatuses form, as a whole, a complete enlarged image. 

29. A storage medium including a program stored thereon for causing a computer to control a display apparatus con- 
nectable with one or more other display apparatuses and including display means for displaying an image, said 
program comprising: 

35 

an image enlarging step, of generating, from the input video signal, an enlarged image of the image corre- 
sponding to the input video signal; 

an authentication step, of performing mutual authentication with said one of other display apparatuses; and 
a display control step of, if the authentication has been successfully passed, displaying an enlarged image 
40 generated by the image enlarging means on the display means such that images displayed on the display 

apparatus and said one or more other display apparatuses form, as a whole, a complete enlarged image. 

30. A display system comprising at least a first display apparatus and a second display apparatus, 

the first display apparatus comprising: 

45 

display means for displaying an image; and 

output means for outputting a video signal to be used by the second display apparatus to display an enlarged 
image, 

50 the second display apparatus comprising: 

input means for inputting the video signal output from the first display apparatus; 

image enlarging means for generating, from the input video signal, an enlarged image of the image corre- 
sponding to the input video signal; 
55 authentication means for performing mutual authentication with the first display apparatus; 

display means for displaying an image; and 

display control means for, if the authentication has been successfully passed, displaying an enlarged 
image generated by the image enlarging means on the display means such that images displayed on the first 
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and second display apparatuses form, as a whole, a complete enlarged Image. 

31. A display apparatus, connectable with one or more other display apparatuses and having display means for dis- 
playing an image in accordance with an input video signal, comprising: 

input means for inputting the input video signal; 

prediction coefficient generation means for generating a prediction coefficient; 

pixel prediction means for predicting the pixel of interest such that plural pixels in the vicinity of the pixel of 
interest are extracted from the input video signal, and the pixel of interest is predicted by means of a prediction 
operation using the extracted plural pixels and the prediction coefficient; and 

display control means for displaying an image including at least the pixel of interest on the display means such 
that images displayed on the present display apparatus and said one or more other display apparatuses form, 
as a whole, a complete enlarged image of the image. 

32. A display apparatus according to Claim 31, wherein the display control means includes 

video signal generation means for generating a video signal to be used to display an enlarged image corre- 
sponding to the input video signal, using the input video signal and the predicted pixel of interest; and 

video signal output means for outputting at least a part of the video signal, to be used to display the enlarged 
image, to said one ore more other display apparatuses. 
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